Julian's Blogtech

We're thrilled to spread the word about the second annual Juicy Ideas Competition, sponsored by AdvantageWest and DigitalChalk!

Google is excited to support the Juicy Ideas competition again this year by hosting the winning team at company headquarters in Mountain View California in Summer 2010, where winners will also each receive an Android-powered phone.

Juicy Ideas 2010 is asking you: "What can you do with data?"

Useful data streams are published by governments, private companies, NGOs, and other services and we want to see how you can harness this data to benefit your community in the form of a software application. Submissions will be judged on their ability to disseminate data and benefit the community, on the usability of the application, and on the business potential of the idea.

This year's contest is open to communities where Google operates offices or datacenters in the US, and universities within 50 km of offices are invited to participate. We've also invited school from last year's Juicy Ideas to join again.

Ready? Form a team of 3-5 students at an eligible college or university.

Set? Research publicly available data, come up with a business plan for that data.

Juice! Create a software application that uses the data to benefit your community, and upload a demo video to Youtube. You'll be judged on the app and the video.

The contest period runs from today through April 11 2010, so get started at www.juicyideas.com!

[From time to time we invite guests to blog about initiatives of interest, and are very pleased to have Allison Schwam, Senior Search Analyst at Backcountry, join us here. -Ed.]

When you don’t have to sacrifice your love of the outdoors for your career or vice versa, it’s something special. In fact, my love of both skiing and marketing has grown dramatically since I took my job at Backcountry. Getting to work with Google, specifically managing our AdWords account, is an online marketing geek’s dream come true. Combine that with every skier’s dream of Utah powder, and life is good.

Day traders wake up every morning to check their portfolio — I get up and check my AdWords accounts. Backcountry sells gear and equipment for the outdoor enthusiast from ski boots to tents, and we sell all of it online. My job is to drive valuable, qualified traffic 365 days a year to Backcountry using AdWords. The AdWords platform lets me manage hundreds of campaigns and hundreds of thousands of keywords with relative ease. I have access to huge amounts of data that are revealed as daily ebbs and flows in impressions, clicks and bids. If you do a Google search for [telemark ski gear], you’ll see our ad:


Backcountry was founded in 1996 by two self-proclaimed ski bums, John Bresee and Jim Holland. Since then, the company has grown to hundreds of employees. I’ve been working here for over two years. Ultimately, our goal is to “crush it,” as some ski town folk say: work hard, play hard.

A typical powder day for me is like this one last Friday when Park City got 12 inches of new snow overnight. Here’s how AdWords helps me manage both work and fun.

7 – 7:45am
Roll out of bed.
Get the coffee going.
Fry eggs and bacon.
Check snow totals.

If it looks like a good ski morning, I first check my email and glance over our AdWords campaigns. All I need to do is my daily reporting to see that I’m on target for my revenue and cost goals. As long as things are okay, I email my boss to say I’ll be out slaying the white dragon.

Just as I have the ideal tools to maximize our online campaign performance, I have the tools avid skiers covet for deep days: fat, rockered skis, stiff ski boots, Gore-Tex jacket and pants, helmet, goggles, merino wool layers, etc. After I grab my gear, I’m off.

7:45am – 12noon

My commute to The Canyons Ski Resort takes 10 minutes. My friends and I know how to get the most out of our time on the mountain, balancing chair lift time, snow quality and vertical. Does that sound a bit like cost-per-click, conversion rate and top-line revenue? Take this lift to that lift, ski the trees while we wait for that chair to open, get after our favorite steep lines. Next thing you know:

Photo by Jim Harris
Face shots are invigorating. Hard to explain, best to experience! After a few glances at the time and collecting my thoughts, I make my way off the mountain.

12pm – 5pm

I head a few miles down the road to the office. As the afternoon goes by, co-workers will emerge from their cubicles; sometimes because legs are cramping up but also to share stories about how the morning was. Where did you ski? How was the snow? Smiles all around.

I settle into work knowing what I need to succeed at my job is at my fingertips. AdWords gives me visibility into my programs to prioritize and understand trends. It also makes it easy to add and edit my account without getting bogged down in manual work. I regularly use Keyword Performance Reports to monitor both head terms and tail terms to stay on top of revenue opportunities. I’ll take into account the average order value and also the percentage of clicks that turn into sales (rate of conversion) in order to manage our keyword bids. As I do bid updates, I also check the AdWords Preview Tool to see how our ads are ranked and what is going on with our competition. We don’t really focus on “cost-per-click” but instead on “cost as percentage of revenue,” which means the more people purchase, the more ads we can run. So the higher the return on advertising spend, the more room we have to grow our paid search presence.

5 pm – 8 pm

I like this afternoon time in particular because it’s quiet and I can really focus on data-crunching. Uninterrupted time and a strong cup of coffee are essential for doing long-term analysis.

Campaign Performance Reports are great for identifying long- and short-term trends as seasons shift or for changes in demand by brand. We just wrapped up our winter sale, so this is a great time for me to run an Ad Performance Report to analyze which versions of ad copy had the strongest clickthrough rates for future reference. Finally, Google Insights for Search is a fun tool. It’s a great way to connect with our buyers by discussing big-picture trends with the brands we carry. We can look at AdWords Campaign performance and try to tie it back to general search volume in the marketplace and identify product searches on the rise. In short, given our metrics focus, AdWords gives me the information I need to make decisions about specific keywords, bids, and our overall spend.

There are typically the same few folks hanging out at the office this late. We’ll exchange some pleasantries, and as the lights get turned off I’ll shut down my computer.

When I earned an undergraduate degree in marketing and cultural anthropology, I had only a vague idea how I was going to create a career with behavioral and analytical activities. It turns out online marketing is an exciting mix of real-time data and customer service. AdWords lets me manage campaigns very efficiently, so I have time to dig deeper and do the strategic analysis that makes this job about much more than just meeting revenue goals.

Finally, I picked Park City because it’s more than a ski town. Here fanatical skiers, trail runners, bikers, snowboarders and climbers can live year round and still have a meaningful career. I’ll always be grateful to companies like Backcountry and Google for making this possible: Backcountry for fostering the passions of the outdoor enthusiast, and Google for innovation in creating the forums and tools that really work for us.

Posted by Allison Schwam, Senior Search Analyst at Backcountry

The Oscars®: glitz, glamor, gossip, gold statuettes, much fanfare — and for many fans, Google search is increasingly a part of watching this live TV experience. Before and during the Academy Awards® broadcast in the U.S., we saw related queries on Google dominating the hot searches list on Google Trends. People searched for the TV schedule, printable ballots for voting on favorites, streaming video sites, nominee and film information, celebrity chatter and whatever else caught your attention. Here's a snapshot:

Fashion
Everyone knows the red carpet is all about the gowns. So which actresses made the best (or worst!) dressed list in search? Zoe Saldana's purple Givenchy haute couture gown won the day in searches, with just a few more queries than runner up Miley Cyrus, who walked down the carpet in one of Jenny Packham's finest. Both beat searches for Sandra Bullock's dress (Marchesa) by large margins. Sarah Jessica Parker, always the fashionista, ranked a distant fourth in Chanel. Certain designers were also popular in search, thanks to the stars who wore (and name-dropped) them. Elie Saab (worn by Anna Kendrick), Armani Prive (Amanda Seyfried, Jennifer Lopez) and Marchesa (Sandra Bullock, Vera Farmiga) were all rising trends.

Winners
Throughout the night Oscar®-related searches rose and fell as nominations were introduced and winners announced. Avatar had the most searches before the ceremony, but as The Hurt Locker received more awards, searches for that film exceeded all others and peaked when it won Best Picture. Precious also had a good run throughout the night. Its peak matched that of "Avatar" during the ceremony:


The awards for best actor, best actress and best director are some of the most-anticipated in the program. There was a considerable amount of buzz about Jeff Bridges, Sandra Bullock and Kathryn Bigelow before last night, and considerable spikes in search volume when they each won. Here's a look at a few of the star searches last night:


Finally, the evening wasn't all about big wins. The awards help expose more obscure films — shorts, documentaries and foreign-language — to a much larger audience. Searches for these titles typically went up tenfold during the evening, and if they took home a gold statue, search volume spiked as much as 100 times higher. Music by Prudence, Logorama, Food, Inc., The Cove and The Secrets in their Eyes all experienced an exponential explosion of queries.

Gossip and memorable moments
People are always eager for more information about the Hollywood stars — personal stats like age, height, family and dating status. During this year’s Oscar® ceremonies, Kathryn Bigelow's height and Miley Cyrus's mother's tattoos were hot topics. Whose girlfriend was most searched for? Easy. George Clooney's (Elisabetta Canalis).

So what were the most memorable moments of the broadcast? When George Clooney wandered off the red carpet to greet the crowd, queries on [clooney] shot through the roof. Ben Stiller’s appearance as a Na'vi was another draw, and queries on him were high during his spoof. Molly Ringwald and Matthew Broderick’s John Hughes tribute triggered a flood of nostalgia; Sixteen Candles, The Breakfast Club, Ferris Bueller's Day Off and Home Alone all saw huge query spikes as well. During the memorial portion of the show, queries surged for those in the industry who passed away in the last year, including Ron Silver, Natasha Richardson, Patrick Swayze and Brittany Murphy. On a lighter note, Sandra Bullock set off a frenetic amount of searches when she mentioned a Meryl Streep kiss in her acceptance speech.

As we've seen in presidential debates, unusual words also generate great interest. The Oscars led to spikes in searches for [catharsis] (from Robert Downey Jr.'s presentation with Tina Fey) and [spooning] (Colin Farrell talking about Jeremy Renner).

We hope you enjoyed the evening, and want to thank everyone for turning to Google search to see the latest. And our parents deserve huge thanks, and our agent... oh, they're telling me to wrap it up...!

Posted by Qing Wu, Senior Economics Analyst

This is part of a regular series of posts on search experience updates that runs weekly. Look for the label This week in search and subscribe to the series. - Ed.

This week's enhancements include:

Stars in search
Every day, we work to improve the four key components of search: comprehensiveness, latency, user experience and relevance. Of these, relevance is dramatically enhanced by more personalized results. This week, we announced a new feature that makes it much easier to mark and rediscover your favorite content. Stars in search are just like the stars you see in Google Toolbar or in Maps — they act like bookmarks. When you star a search result, and it happens to appear again in future results, you'll see that you already found that particular result. Starred items will appear at the top of your results. Stars in search has been rolling out this week, and will be available globally for all users who are signed in to their Google account.

Auto-spell for images
This week, we unveiled automatic spell correction for images. In cases where we're highly confident you had intended to type something else, we'll replace results from the typo query with those from the spell-corrected version — just like when you misspell a query in Google search. Ultimately, this change will reduce the time it takes to get you the result you're looking for (and that's a good thing).

Example searches: [butterflys], [roman architecture] and [apollo ohno]

Sidewiki page owner entry
In September, we launched Sidewiki, which lets you contribute helpful information to any webpage using a sidebar in Google Toolbar or a Chrome extension. Afterward, webmasters asked, "How can I quickly put Sidewiki on all pages of my site?" Now webmasters can create a special entry, called a page owner entry, that appears above all entries written by users. Webmaster tool improvements ultimately create a better web experience for us all, so we're pleased about this. Let us know what you think about our webmaster enhancements.



Stay tuned for next week's news on more search launches.
Posted by Johanna Wright, Director of Product Management, Search

Google Summer of CodeTM, our flagship program to introduce college students to open source development, opens today. Over the past five years, we've seen more than 3,400 successful students "graduate" from the program, and we're looking forward to welcoming another group of students for our sixth year. We're now accepting applications from open source projects who wish to act as mentoring organizations and will begin accepting applications from students on March 29th. For more details, check out the Google Open Source Blog.

By Leslie Hawthorn, Open Source Team

Looking for new contributors and fresh perspectives for your open source software project? Through the Google Summer of Code™ program, we fund students worldwide to work with mentors from the FLOSS community on a three month coding project. Over the past five years, we've successfully paired nearly 3,400 students "with more than 3,000 mentors from backgrounds spanning industry to academia, with some spectacular results: more than 8 million lines of source code produced and over $20M in funding in support of open source development. We're particularly excited by the social ties our students form through the course of the program. We've connected people in more than 100 countries, and hope to bring people from even more places into the Google Summer of Code community this year. We're looking forward to our sixth year and welcoming another group of 1,000 student developers to the program.

We're now accepting applications from open source projects who wish to act as mentoring organizations. We'll be taking mentoring organization applications until Friday, March 12th at 23:00 UTC. Our list of approved organizations will be published on the 2010 Google Summer of Code site on March 18th. Interested students will then have several days to discuss their ideas with the accepted organizations before student applications open on March 29th.

Check out our Frequently Asked Questions page for more details and a preview of the application. And remember, if you have any questions, you can always find us in the Google Summer of Code Discussion group or in #gsoc on Freenode. Best of luck to all of our applicants!

By Leslie Hawthorn, Open Source Team

(Cross-posted from the Google.org Blog)
What do tracking flu, helping consumers monitor their home electricity use, slowing deforestation and perhaps most importantly in 2010, helping the people of Haiti have in common?

While they are all part of the wide-ranging work of Google.org over the last year, they also show what our technical teams can accomplish in critical areas that don't always get the attention they need and deserve.

A year ago we outlined our goals for the next chapter for Google.org. We talked about our vision to use strengths of Google in information and technology to build products and advocate for critical policies that address global challenges. Ideas for projects continue to pour in from Googlers and partners around the globe, and we're incubating several new projects in the areas of economic development, clean energy and access to technology.

This past year, we:

• Ramped up Google PowerMeter to help consumers reduce their electricity use and save money, secured utility and device partners, and launched the API on code.google.com to help expand partner access globally.
• Introduced Earth Engine, a new computational platform we have begun building for global-scale analysis of satellite imagery to monitor changes in key environmental indicators like forest coverage, at COP15 in December.
• Quickly expanded Google Flu Trends to 20 countries and 38 languages as the H1N1 flu virus spread around the world. We also added city-level flu estimates to 121 U.S. cities and developed the Flu Shot Finder to help people find vaccine locations.
• Responded to earthquakes in Haiti and Chile, with maps, updated earth imagery, and networking projects, and built Person Finder to help people find information about their loved ones after a disaster.
• Advocated for policies to spur innovation of renewable energy technologies that are cheaper than coal (RE<C), and our engineers worked on ways to reduce the cost of solar thermal and other RE<C technologies.

We will continue to greenlight large scale engineering projects that build on Google's strengths in technology, our computing infrastructure and global teams.

Overall, our philanthropic mission at Google includes our Google.org projects and a range of other initiatives — from grants, scholarships and other charitable giving programs to in-kind product support for non-profits. Our founders have set a goal of devoting approximately 1% of Google's equity and yearly profits to philanthropy. In 2009, we devoted around $100 million plus in-kind giving to a broad range of philanthropic efforts. Here are some highlights:

• Academic scholarships and awards: We provide scholarships to encourage students of various backgrounds, ethnicities and gender to excel in their studies in hopes that these and other programs will help dismantle barriers that keep women and minorities from entering computing and technology fields.
• Academic grants: We support the next generation of engineers and maintain strong ties with academic institutions worldwide that are pursuing research in core areas relevant to our mission. We fund projects across a variety of subjects, host visiting faculty members at Google, and have launched the Google Fellowship Program to fund graduate students doing innovative research in several fields.
• Holiday charitable gift: We made $22 million in donations in 2009 to a couple of dozen deserving charities around the world to help organizations that have been stretched thin by more requests for help in a year of fewer donations.
• Employee gift matching: Google matches up to $6,000 for each employee's annual charitable contributions and contributes $50 for every five hours an employee volunteers through our "Dollars for Doers" program to encourage employee participation in charitable causes.
• Charitable Giving Council: We support grants for Googler-led partnerships on causes such as K-12 educational initiatives in science, math and technology.
• Community affairs: We invest in communities where Google has a presence around the world, creating opportunities for Googlers to invest time and expertise, engage in local grant making and build partnerships with local stakeholders.


In addition, our Google for Non-Profits site provides information and links to free tools to help charitable groups promote their cause, raise money, collaborate with others and operate more efficiently. Google Grants, for example, offers in-kind AdWords advertising to non-profit organizations. Since the program began, we've donated over $625 million worth of AdWords advertising to all kinds of charitable organizations.

To keep up with our activities, check out the Google.org blog.

Posted by Megan Smith, VP and General Manager, Google.org

This post is part of the Who's @ Google I/O, a series of blog posts that gives a closer look at developers who'll be speaking or demoing at Google I/O. This post is written by Albert Wenger, partner at Union Square Ventures (and still enjoys writing code!). Albert will be speaking alongside others in venture capital on a panel at Google I/O.

Reading the Google Code blog, it is hard not to marvel at the fundamental transformation that is taking place in the business of code. By the business of code, I mean the economics of developing and selling software. My first exposure to the software business was as a teenager in Germany some twenty five years ago. Driver's education there is quite expensive because one has to take many mandatory lessons. After all, once you have passed you get to drive on the Autobahn, which, to this date, has long stretches without a speed limit! I thought I was being clever by agreeing to write software for a driving school in exchange for free lessons. It turns out the clever one was the owner of the driving school who turned around and sold the software to several other schools.

So what would it have taken for me then to become an ISV (independent software vendor), other than actually having the idea? These were the early days of PCs. I would have had to spend a lot of money on marketing and a lot of time on in-person sales and on-premise / telephone support. Most ISVs at the time grew locally for that reason and it was not uncommon to have highly fragmented markets with literally dozens of different vendors. As the software would have grown past the very simple initial functionality that I had created, I would have had to write pretty much everything I needed myself. Need billing? Write a billing module. Need asset tracking? Write an asset tracking module. The marketplace for components evolved only much later and was almost as fragmented as the ISV market.

This situation persisted until quite recently. In fact, in 2003 I spent a year getting to know the market for software for trucking companies in the US (Why? That's a long story). That market still had essentially the same characteristics: highly fragmented, regional customer bases, and almost 100% monolithic custom systems.

Since then, the situation has changed dramatically. Today, creating new software means focusing on what the unique contribution is that one wants to offer and figuring out how to integrate with everything else. Need a spreadsheet? Use Google Apps. Need telephony? Use Twilio (disclosure: Twilio is a USV portfolio company). Marketing can happen on the web through keyword advertising and, better yet, SEO and customer sharing. For many solutions, even sales can be entirely web-based (enter a credit card!). Support can happen over the web and often users can support each other through community. Add cloud computing to the mix and you eliminate the fixed cost that was such a high barrier to entry in the early days of the web (I remember the extraordinary bills for servers and bandwidth in 1999!).

All of this has made it possible for small teams to create big successes. It is amazing what a few great coders can do, leveraging all the services that are now available. It is,however, not just the cost side of the business of code that has changed dramatically. Competition has gone global. Someone in a faraway place can create a system, and it is instantly available everywhere. The days of the profitable, regional ISV business are over. The source of competitive advantage has also shifted. In the past, if your solution had better features, you could land a sale even against a competitor that had more customers. Now, better features don't mean that much if the larger competitor has built a network effect into their business. Imagine trying to start a LinkedIn or Salesforce competitor with better features. So the very same forces that are making it much easier to get started are making it much harder to build a successful and sustainable business.

Does that mean that there will be fewer opportunities going forward? Maybe. But there is a strong countervailing force that is creating important new opportunities: the code of business is also changing. By the code of business, I mean how companies and industries (and even societies) are organized. At Union Square Ventures, we are convinced that over time, the Internet will transform most, if not all, industries as much as it is changing the software business. This has started with the media industry, where after years of prediction of change we are now seeing massive shifts.

The reason that the code of business will change is that much of it is based on historic constraints on the bandwidth and latency of information flows. For instance, a command-and-control type hierarchy is still at the heart of (almost) all large corporations. Information flows up the hierarchy with middle management in charge of aggregating information flows. Commands then flow down with middle management translating into finer grained actions. This basic structure dates back to a time of messengers and telegraphs. Corporations are slowly shifting away towards more of an Internet architecture of "small pieces, loosely joined" -- but in many cases the end state may mean that the "pieces" are independent, small companies instead of units of a large company.

These changes will take a great deal of time (decades) because existing structures have a ton of inertia. Far more people tend to be interested in preserving the status quo than in making radical changes. Also, when a whole system needs changing, it is often difficult to get there one piece-at-a-time because all the components need to fit together. But as they start to occur in other industries, the opportunities will be massive. To give just one example, consider education: the size of the textbook industry alone in the US is estimated at $7 billion annually. This is a pure content business ripe for disruption.

Enough reading -- time for everyone with a transformative idea to start coding!

By Albert Wenger, Union Square Ventures and Continuations.

Spring is almost here - and that means time for March Madness, Spring Break, and of course..midterms! Are you looking for help with your CS courses or just want to explore new and exciting content? Check out Google Code University where you can find tutorials, lectures, and problem sets for current computing technologies and paradigms. The content - developed by some of the best schools and professors around the US - is cutting edge. And since all of it is Creative Commons licensed, you can even download it and reuse it in your class.
Check out some of the new additions to Google Code University:

• Android Development
• Python Programming

We are always looking for fresh content - do you know a professor would be a great contributor? A class that you thought could help other students like you around the world?Submitting course content on Google Code University is easy! Just have your professor visit the content submission page and submit an outline of their idea - we'll take it from there.
Good luck!
Posted by Mary Radomile, Education Program Manager

Last year, we released a public data search feature that enables people to quickly find useful statistics in search. More recently, we expanded this service to include information from the World Bank, such as population data for every region in the world. More and more public agencies, non-profits and other organizations are looking for ways to open up their data and expand global access to this kind of information. We want to help keep that momentum going, so today we're sharing a snapshot of some of the most popular public data search topics on Google. We're also launching the Google Public Data Explorer, an experimental visualization tool in Google Labs.

Popular public data topics on Google
We know people want to be able to find reliable data and statistics on a variety of subjects. But what kind of statistics are they looking for most? To help us better prioritize which data sets to include in our public data search feature, we've analyzed anonymous search logs to find patterns in the kinds of searches people are doing, similar to the patterns you can find on Google Trends and Insights for Search. Some public data providers have asked us to share what we've learned, so we decided to put together an approximate list of the 80 most popular data and statistics search topics.

You can read the complete list at this link (PDF), but here's the top 20 to get you started:
1. School comparisons
2. Unemployment
3. Population
4. Sales tax
5. Salaries
6. Exchange rates
7. Crime statistics
8. Health statistics (health conditions)
9. Disaster statistics
10. Gross Domestic Product (GDP)11. Last names
12. Poverty
13. Oil price
14. Minimum wage
15. Consumer price index, inflation
16. Mortality
17. Cost of living
18. Election results
19. First names
20. Accidents, traffic violations
You'll notice some interesting entries in the list. For example, we were surprised by how many people search for data about popular first and last names. Perhaps people are trying to decide what to name a new baby boy or girl? As it turns out, people are interested in a wide range of statistical information.

To build the list, we looked at the aggregation of billions of queries people typed into Google search, using data from multiple sources, including Insights for Search, Google Trends and internal data tools — similar to what we do for our annual Zeitgeist. We combined search terms into groups, filtering out spam and repeats, to prepare a list reflecting the most popular public data topics. As a statistician, it's important for me to note that the data only covers one week's worth of searches in the U.S., so there could be seasonal and other confounding factors (perhaps there was an election that week). In addition, preparing a study like this requires a fair amount of manual grouping of similar queries into topics, which is fairly subjective and prone to human error. While imperfect, we still think the list is helpful to consider.

The Public Data Explorer
As you can see, people are interested in a wide variety of data and statistics, but this information is only useful if it's easy to access, understand and communicate. That's why today we're also releasing the Google Public Data Explorer in Labs, a new experimental product designed to help people comprehend data and statistics through rich visualizations. With the Data Explorer, you can mash up data using line graphs, bar graphs, maps and bubble charts. The visualizations are dynamic, so you can watch them move over time, change topics, highlight different entries and change the scale. Once you have a chart ready, you can easily share it with friends or even embed it on your own website or blog. We've embedded the following chart using the new feature as an example:



This chart compares life expectancy and the number of births per woman over the last 47 years for most economies of the world. The bubble sizes show population, and colors represent different geographic regions. Press the play button to see the dramatic changes over time. Click "explore data" to dig deeper.

Animated charts can bring data to life. Click the play button in the chart to watch life expectancy increase while fertility rates fall around the world. The bubble colors make it quick and easy to see clusters of countries along these variables (e.g., in 1960 the European and Central Asian countries were in the lower right and Sub-Saharan Africa in the upper left). The bubble sizes help you follow the most populous countries, such as India and China. These charts are based on the Trendalyzer technology we acquired from the Gapminder Foundation, which we've previously made available in the Motion Chart in Google Spreadsheets and the Visualization API.

With a handful of data providers, there are already billions of possible charts to explore. We currently provide data from the same three providers currently available in our search feature: the World Bank, the U.S. Bureau of Labor Statistics and the U.S. Census Bureau. In addition, we've added five new data providers: the Organization for Economic Co-Operation and Development (OECD), the California Department of Education, Eurostat, the U.S. Center for Disease Control, and the U.S. Bureau of Economic Analysis. We're excited that all around the world new data providers are deciding to make their information freely available on the Internet, enabling innovators to create interesting applications, mash up the data in new ways and discover profound meaning behind the numbers.

We hope our list and new tool help demonstrate both the public demand for more data and the potential for new applications to enlighten it. We want to hear from you, so please share your feedback in our discussion forum. If you're a data provider interested in becoming a part of the Public Data Explorer, contact us.

Posted by Jürgen Schwärzler, Statistician, Public Data team

I'll be a presenting a paper at ICST 2010 in Paris April 6-10 about how Google tests and builds software. Here's a pointer to the program if you are interested. Also here's a link to the abstract of the talk itself. I'll publish the paper after the talk here. Hopefully, I'll see some of you there!
Posted by Patrick Copeland

Next week several Googlers will be attending and presenting at the 41st ACM Technical Symposium on Computer Science Education (SIGCSE 2010). From March 10-13, Leslie Hawthorn and Cat Allman from the Open Source Programs Office will be in Milwaukee, WI, USA to talk about Google’s open source student programs, Google Summer of Code™ and the Google Highly Open Participation Contest. Check out Google’s vendor session on Friday to hear more from Leslie and Cat. Leslie will also be speaking at a roundtable and panel discussion with Hal Abelson from the Google App Inventor team at the Humanitarian FOSS Symposium on Wednesday.

If you are interested in learning more about Google’s activities in computer science education, make sure to attend some of the talks we have scheduled or drop by the Google booth!

By Ellen Ko, Open Source Team

The Google Open Source Team has the privilege of funding some really great projects in the Open Source space. Mathieu Desnoyers, a student at Ecole Polytechnique, recently defended his Ph.D. thesis, which we helped to fund. The topic of his thesis was "Low-Impact Operating System Tracing."

The open source projects he created as part of his work were two-fold: Linux Trace Toolkit Next Generation (LTTng), a LGPLv2.1/GPLv2 tracer for the Linux kernel; and Userspace RCU library (liburcu), a highly-scalable user-space synchronization library, distributed under the LGPLv2.1 license.

Mathieu was kind enough to send us this summary of his research:


Computer systems, both at the hardware and software-levels, are becoming increasingly complex. Tracing is the key to solving some or all of this increasing complexity. In the case of Linux, used in a large range of applications, from small embedded devices to high-end servers, the size of the operating system kernels are increasing, libraries are being added, and major redesign of existing software is required to benefit from multi-core architectures. As a result, the software development industry and individual developers are facing problems whose resolution requires an understanding of the interaction between applications and all components of an operating system.

In my thesis, I propose the LTTng (Linux Trace Toolkit next generation) tracer as an answer to the industry and open source community tracing needs. The low-intrusiveness of the tracer is a key aspect of its usefulness because we need to be able to reproduce problems occurring in normal conditions. In some cases, users leave tracers active at all times in production, which makes the tracer overhead definitely critical. Our approach involves the design of synchronization primitives that meet the low-impact requirements. The linearly scalable and wait-free RCU (Read-Copy Update) synchronization mechanism used by the LTTng tracer fulfills these requirements with respect to data read. A custom-made buffer synchronization scheme is proposed to extract tracing data while preserving linear scalability and wait-free characteristics.

By measuring the LTTng impact, I demonstrate that it is possible to create a tracer that satisfy all the following characteristics: low latency, deterministic real-time impact (wait-free), small impact on operating system throughput and linear scalability with the number of cores. Experiments on various architectures show that this tracer is portable.

I propose a general model for superscalar multi-core systems with weakly-ordered memory accesses to perform formal verification of the RCU correctness and wait-free guarantees by model-checking. The LTTng
buffering scheme is also formally verified for safety and progress. Formal verification demonstrates that these algorithms allow reentrancy from multiple execution contexts, ranging from standard thread to non-maskable interrupts handlers, allowing a wide instrumentation coverage of the operating system.


Many thanks to Mathieu for sending us this report. You can download the full dissertation for more details.

By Carol Smith, Open Source Team

When it comes to code, Google's support has made a big difference to the Tor Project. Providing privacy and helping to circumvent censorship online is a challenge that keeps our software developers and volunteers very busy. The Google Summer of Code™ brings students and mentors in the open source community together to write code for three months every year. A lot of coding got done in a few months in 2009, and Tor was lucky to get a group of students who kept on working past the summer months to improve existing projects and support users. Tor also works on Libevent with Google.

All of these changes in software are very exciting, but who is it all for? Why is anonymity online so important? Companies like Google have privacy and opt-out policies, but not everyone has this stance. Corporations, nations, criminal organizations and individuals want your information. Companies collect information on your web browsing habits and sell it or are sloppy when it comes to protecting it from identity thieves. Others can threaten lives, from repressive nations tracking down outspoken journalists, to abusive spouses or stalkers who want to find out where their victims are hiding; from enemy military forces trying to find a communications link, to criminals who know when law enforcement is watching online.



Political upheaval sparks protests and renewed efforts to control the flow of information online. Interest in censorship circumvention also rises. In 2009, use of Tor increased, as users tried to get around national firewalls during the elections in Iran, and after the introduction of national Internet filters in other countries.


In times of relative political stability, governments routinely filter out international news outlets, information on reproductive health, religion, human rights and other topics deemed unfit. Women blogging about things considered mundane elsewhere, like being forbidden to drive or shop alone, are harassed by authorities. On the one hand, technology has made it easier to crack down on dissent, but the right technology can influence policy in good ways. In Mauritania, the use of censorship circumvention software after 2005 became widespread enough to prompt the government to stop filtering, since it was becoming a waste of time.

Even people living in countries where free speech is protected by law need anonymity for political activities. People blogging about political views that differ from the prevailing attitudes in a small community may lose a job or face boycotts if they run a business. In a company town, writing about the misdeeds of the company that employs your neighbors may be dangerous. Telling people about corruption could lead to harassment from guilty officials.

When someone finds the courage to leave an abusive relationship, the support of victims' advocates is vital. The Internet can help a survivor find counseling, shelter, and encouragement from people who have gone through the same process. Sadly, stalkers are also using technology to find their victims. Abusers monitor web browsers to see if a victim is planning to leave. Information about a shelter's location can be found in email headers, forcing abuse survivors to relocate. According to the U.S. Bureau of Justice Statistics, over one in four people who are stalked experience some sort of cyberstalking. Though some software in a stalker's toolkit is installed on a home computer, IP addresses can reveal which internet cafe or library someone uses to get online. Even if you don't have a stalker, hiding your IP address can be a good idea. Kids and adults alike are advised not to tell strangers where they live, but an IP address can reveal it for them.

Sting operations fail if criminals can tell that the police are connecting to message boards and chat from a government network. The information disappears. Insurgents may be looking for soldiers connecting to their defense department's computers back home. Anonymous tip lines are not so anonymous if someone telling authorities about crime is the only person in the neighborhood connecting to a government website. Without anonymity, going after organized crime can be dangerous to officers and their families.

Some companies do not reveal how much they know about their customers, or who sees the information. Some Internet Service Providers feel entitled to sell data collected from their subscribers to marketers. Though they claim that the information is not tied to any particular users, it is easy to find someone based on their search history. Information about visits to banking websites, searches for details on pre-existing health conditions, or other sensitive online activity could be damaging in the wrong hands; whether made available through carelessness or commercial interest.

Privacy online can protect people offline whether they are organizing protests, covering the news, blowing the whistle on threats to public health, or just blogging about daily life. In the "real world" assaults on privacy like peeking in windows, opening mail, or breaking and entering are obvious crimes. In the online world, however, assaults on privacy are subtle and unyielding. These threats to your health, your wealth and your well-being have no "opt-out" button. They have no "scrub my data" option. Your online activities, e-mails, bank transactions and everything else can be used to trace where you are and who you are. Using software like Tor gives ordinary citizens more choice about the information they reveal online.

For more information about online privacy and circumventing internet censorship, visit the Tor Project's website.

By The Tor Project

By James A. Whittaker
Flashback. It's 1990. Chances are you do not own a cell phone. And if you do it weighs more than a full sized laptop does now. You certainly have no iPod. The music in your car comes from the one or two local radio stations that play songs you can tolerate and a glove box full of CDs and cassettes. Yes, I said cassettes...you know the ones next to those paper road maps. Music on the go? We carried our boom boxes on our shoulder back then.
If you are a news junkie, you get your fix from the newspaper or you wait until 6 ... or 11. Sports? Same. Oh and I hope you don't like soccer or hockey because you can't watch that stuff in this country more often than every four years. Go find a phone book if you want to call someone and complain.
I could go on, and on, and on, but you get the point. Oh wait, one more: how many of you had an email address in 1990? Be honest. And the people reading this blog are among the most likely to answer that affirmatively.
The world is different. The last 20 years has changed the human condition in ways that no other 20 year period can match. Imagine taking a 16 year old from 1990 and transplanting him or her to a 2010 high school. Culture shock indeed. Imagine transporting a soccer mom, a politician, a university professor... Pick just about any profession and the contrast would be so stark that those 1990 skills would be a debilitating liability.
Except one: that of a software tester. A circa 1990 tester would come from a mainframe/terminal world. Or if they were on the real cutting edge, a locally networked PC. They'd fit into the data center/slim client world with nary a hiccup. They'd know all about testing techniques because input partitioning, boundary cases, load and stress, etc, are still what we do today. Scripting? Yep, he'd be good there too. Syntax may have changed a bit, but that wouldn't take our time traveler long to pick up. That GEICO caveman may look funny at the disco, but he has the goods to get the job done.
Don't get me wrong, software testing has been full of innovation. We've minted patents and PhD theses. We built tools and automated the crud out of certain types of interfaces. But those interfaces change and that automation, we find to our distress, is rarely reuseable. How much real innovation have we had in this discipline that has actually stood the test of time? I argue that we've thrown most of it away. A disposable two decades. It was too tied to the application, the domain, the technology. Each project we start out basically anew, reinventing the testing wheel over and over. Each year's innovation looks much the same as the year before. 1990 quickly turns into 2010 and we remain stuck in the same old rut.
The challenge for the next twenty years will be to make a 2010 tester feel like a complete noob when transported to 2030. Indeed, I think this may be accomplished in far less than 20 years if we all work together. Imagine, for example, testing infrastructure built into the platform. Not enough for you? Imagine writing a single simple script that exercises your app, the browser and the OS at the same time and using the same language. Not enough for you? Imagine building an app and having it automatically download all applicable test suites and execute them on itself. Anyway, what are you working on?
Interested? Progress reports will be given at the following locations this year:
Swiss Testing Day, Zurich, March 17 2010
STAR East, Orlando, May 2010
GTAC, TBD, Fall 2010
Here's to an interesting future.

This year's conference is now sold out, which means we'll be seeing over 4,000 of you on May 19-20 at Moscone West! For those of you who can't join us in person, video recordings of all sessions and keynotes will be available on YouTube following the conference.
Continue to follow us on Twitter for updates on sessions, speakers and the Sandbox. We'll also continue posting updates and Google I/O-relevant content on this blog.
By Joyce Sohn, Google Developer Team

As of today, this year's Google I/O conference has sold out and registration is closed. That means more than 4,000 developers will be joining us on May 19-20 at Moscone West in San Francisco.

Like years past, I/O will feature over 90 in-depth sessions and the opportunity to meet and learn from other developers, including those from the more than 160 companies that will demo in the Developer Sandbox. For those unable to attend, video recordings of technical sessions will be available on YouTube following the conference.

From now until May, we'll continue to list new speakers, new sessions, and new Sandbox participants on the Google I/O website. To keep up with the latest event info and details, follow us on Twitter.

Posted by Vic Gundotra, Vice President of Engineering

Posted by Sanjiv Kumar and Henry Rowley, Google Research

A few weeks ago we announced the launch of a new orkut application in Google Labs called People Hopper that lets you take your profile image and "morph" it into a friend's photo, using publicly available images from other orkut users along the way. No computer graphics tricks are used; every image along the transition comes from real orkut users.



The application hops across millions of public user images in orkut so that one image is smoothly transformed into another. First, faces are automatically detected in public profile images and normalized in contrast and size. Then, for each image, we find other public profile images that are similar to it. Finally, when you pick two faces, we just hop between similar public images, step-by-step, until the connection is made.

People Hopper was outcome of the following research question: Is it possible to learn a low-dimensional space (i.e. a manifold) in which all the human face images live? It is well-known in the machine learning community that to recover the true underlying manifold one needs a large number of samples from it. In 2008, we published a paper at CVPR in which we learned a face manifold using tens of millions of images, which is still the largest scale manifold learning study to date.

To be able to do manifold learning at such a large scale, we had to address two key issues: First, how to do nearest neighbor search in very large databases? We used spill-trees to speed up the search to construct the neighborhood graph. Second, how to do spectral decomposition of matrices which are hundreds of terabytes in size? We investigated sampling-based matrix decomposition methods to handle such matrices.

One way to visualize the quality of the manifold is to find shortest paths between pairs of faces in the manifold, and observe the smoothness of the transitions between them. This is exactly what People Hopper does. Curious? Try People Hopper on orkut now!

The quality of the face manifold depends on three main factors: the number of faces in the manifold, the appearances of those faces, and the similarity measure used for image matching. Since we cannot control the number or appearance of the faces in orkut profiles, it may happen that for a particular image there exists no visually similar image in the database. We plan to update our graph over public profile images frequently, so the quality of paths will change as users join orkut or update their profile images. Finding better contrast normalization and similarity measures is a topic of continuing research. Currently we don't use any face-specific features during this process, just simple image distances.

We are eager to hear your feedback on how we can make this application more fun and useful. Also, if for any reason you would prefer your profile image not to appear in any People Hopper path, you can choose to opt out by visiting our People Hopper homepage.

We've long believed that personalization makes search more relevant and fun. For nearly five years, we've been tailoring results with personalized search. Today we're announcing a new feature in search that makes it easier for you to mark and rediscover your favorite web content — stars.

With stars, you can simply click the star marker on any search result or map and the next time you perform a search, that item will appear in a special list right at the top of your results when relevant. That means if you star the official websites for your favorite football teams, you might see those results right at the top of your next search for [nfl]. Here's what the new "Starred results" feature looks like:


The great thing about stars is that you don't have to keep track of them. You don't even have to remember whether or not you starred something. Simply perform a search and you'll rediscover your starred items right when you need them. Stars sync with your Google Bookmarks and the Google Toolbar, so you can always see your list of starred items in one place and easily organize them. Even beyond the results page, while browsing the web you can quickly click the star icon in Toolbar to create a bookmark, and those pages will start showing up in the new stars feature.

Stars in search replace SearchWiki. In our testing, we learned that people really liked the idea of marking a website for future reference, but they didn't like changing the order of Google's organic search results. With stars, we've created a lightweight and flexible way for people to mark and rediscover web content. For people who like annotations, we have Sidewiki, a more powerful way for people to contribute and discover helpful information next to pages across the Internet. All your existing SearchWiki edits will be preserved with your Google Account. You can learn more on our help center.

Stars in search are rolling out in the next couple days and will be available globally for all signed-in users.

Posted by Cedric Dupont, Product Manager and Matthew Watson, Software Engineer

This post is part of the Who's @ Google I/O, a series of blog posts that give a closer look at developers who'll be speaking or demoing at Google I/O. This guest post is written by Seth Priebatsch, Chief Ninja of SCVNGR, who's creating a mobile game for the conference.

SCVNGR is a platform for quickly and easily building location-based mobile games. Each game is all about doing challenges at places. Go here and take a photo, go there and solve this riddle. You happen to be at this coffee shop? Awesome! Try this challenge and earn a couple points! SCVNGR powers games for all sorts of institutions ranging from Princeton to Harvard to the Smithsonian Institutes to SIGGRAPH and even the U.S Navy.

If you're attending Google I/O this year, you'll get to try out our mobile game at the conference! (Don't forget to bring your Android phone, if you've got one!) I'm not going to give it all away here, but I do want to talk about one of the especially cool features that we're rolling out using some neat Google APIs.

One of the biggest challenges that our game-builders face is how to build location-based challenges that truly verify the user has actually made it to the right location. There are some non-technical solutions to this problem, such as creating riddles that require the user to be there to solve them (i.e. what is the third word on the fourth line of the plaque on the back wall) or taking photos which are then verified manually by the community or the game developer themselves.

We've also looked at a number of more technical solutions. The most obvious being to take the geo-tagged coordinates of each of the locations with a game and then use GPS to ensure that the player is within a certain radius of the location. Unfortunately, GPS verification has issues when the locations are indoors (as many are) and can vary greatly in accuracy across difference devices.

A new option, one that we're launching in a couple of weeks, uses QR codes planted at locations within the game-board. Players must scan these QR codes to verify that they've made it to the right spot. We're using the Google Charts API as an easy way to programmatically generate QR codes. Of course, generating and planting the QR codes is only half of the equation. You've also got to be able to decode them from the phones during the game. We experimented with a couple of options as to how to best achieve this.

Our first thought was to simply have the players snap pictures of the QR code which we'd then post back to our server, decode and respond with whether or not that was in fact the right QR code (or a QR code at all). The benefit of this solution was only having to utilize one QR code processing library for both our iPhone and Android applications. But we ran into a couple issues right up front:

1. The time cost incurred by having to transmit a reasonably high-resolution image to our servers.
   
2. Most players aren't very good at taking pictures of QR codes. They move the lens of the camera very close and snap the picture. (It's actually best to take the picture from 12-24 inches away to enable the camera to focus sharply on the QR code.) This led to a high-number of failed submissions before we were actually able to recognize a valid QR code.

Add all this up and it created a pretty poor game-play experience.

So we turned to the ZXing project (pronounced Zebra Crossing) which is an open source barcode processing library written in Java and highly suited to the Android environment. Running the ZXing code right on the device rid us of the time-delay introduced by transmitting images to the server. But we still had the issue of the high-fail rate of the user snapping unsuitable images. Rather than trying to implement any form of "auto-scanning", we've chosen to simply grab images from the camera every 1/8 of a second or so, scan them and stop the process once a QR code is recognized.

As for the iPhone, ZXing has an objective-c port for the iPhone, but in order to grab images in real-time from the camera, you'll have to use a private API call for iPhone OS 3.1. Luckily, Apple has officially authorized its public usage until it's made public.

We're hard at work integrating QR codes into the great game that we're building for Google I/O and we hope you'll get a chance to play. The I/O team will let you know when the game is ready! The SCVNGR team will also be there in person, so please come by and say hello! We'd love to get your thoughts on the game or just chat about some of the Google APIs that we've used within SCVNGR.

By Seth Priebatsch, Chief Ninja of SCVNGR