Friday, January 21. 2011
You may recall we wrote a story just after the Christmas holidays about the "Increasing Uselessness of Google Search" - well, it got so far as being mentioned on TWIG (where Jeff Jarvis roundly dissed us for not knowing what we were talking about*, despite a lot of agreement by ordinary users that they too had experienced a similar thing). Well, today, Google admitted - in their own roundabout "there was no problem, and here are some changes to fix the non-problem" way, that there was a problem:
First, the "there is no problem" bit - Matt Cutts:
Tut - silly users - the algorithm says there is No Problem, so....anyway, then comes the "so now we are fixing the problem that doesn't exist" bit:
As we’ve increased both our size and freshness in recent months, we’ve naturally indexed a lot of good content and some spam as well. To respond to that challenge, we recently launched a redesigned document-level classifier that makes it harder for spammy on-page content to rank highly. The new classifier is better at detecting spam on individual web pages, e.g., repeated spammy words—the sort of phrases you tend to see in junky, automated, self-promoting blog comments. We’ve also radically improved our ability to detect hacked sites, which were a major source of spam in 2010. And we’re evaluating multiple changes that should help drive spam levels even lower, including one change that primarily affects sites that copy others’ content and sites with low levels of original content. We’ll continue to explore ways to reduce spam, including new ways for users to give more explicit feedback about spammy and low-quality sites.
While I hear all this, and am heartened by it, I do believe there is a little bit of trying to tell us that we didn't see what we saw. To reiterate what we observed empirically, in late 2010 and early 2011, if you clicked on something monetisable (like buying a consumer product) you found that most of Page One would lead to sites that basically served more links and Ads, and others that basically copied Wikipedia text and links and Ads, and I wrote the article because my family (and some friends) also remarked on it, and judging by the response a lot of other people noticed it as well. (searching for say "Byzantine Iconoclasts" does not contain spam, but then there is b*gger all money in that, these days anyway). So, I am delighted that these steps are being taken, but I suspect that Google's attention has been refocussed in early 2011 by what was, essentially, a massive user outcry. I have just searched again for the things I was looking for in November/December, and there is a marked improvement today - seeing the camera brand name's website up first again is soooo 2006 :-0
So while I am heartened, I am pretty certain these algorithms have mainly come in very late 2010/early 2011, and may I hypothesize that it was mainly due to the user outcry - and I would also hypothesize that this is why I am reading Matt Cutt's post on 21 January 2011, not say 07 January 2011
Also, perish the thought that we may wonder if they may have been making money in this (potentially) symbiotic relationship:
To be crystal clear:
Well, whatever the reason, I am delighted at the changes made so far, and I look forward to proof positive that they are not making money from the spamsites. A rapid deterioration again in a few months would be a "thing that made me go "Ummm".
*In fairness, he then went on a rant about Buzz, which I enjoyed
Techmeme is now trawling Twitter as well as blogs and the online news world to get an indication of what is being talked about - Wired:
On Thursday, Techmeme chief Gabe Rivera announced that the site would now be aggregating tweets — the 140-character micro-posts found on Twitter.
Why do it, and why now - after all, Twitter has been around awhile? I think the answer is here:
Rivera said he’s not sure if the move will boost traffic, but he said the goal of the shift is to improve the site, “which should expand our readership.”
Au contraire, I suspect the issue Techmeme has been having is that people are linking direct from their Twitter aggregators to stories, rather than going to Techmeme first, a trend which could eventually make one's Twitter aggregator the "Go To" news site (and which is what Tweetmeme et al are trying to do.) If you have the real time search and aggregation already, adding Twitter is not hard to do*
So are blogs therefore dead, as Business Insider argues - on Twitter?
“Now it’s official. Blogs are dead.”
No, we would argue, and that twt shows why - its a statement, without any reasoning or backup. I for one am not going to use a service that just points me to one liners like that. There has to be some beef somewhere, and blogs are the best way of delivering that.
*A bit of an aside - we actually built a blog/news search system a few years ago (see here), and looked at turning it into a Twitter search (it's not that hard) but were darned if we could see a sustainable business model for just reading twts.
Thursday, January 13. 2011
The meme of Google's sputtering search product has certainly taken off, and that of course brings the apologistas out of the woodwork - one of the most amusing is this one on MediaBeat, which somewhat circularly argues that:
I saw it coming first:
You are all wrong, Google is working on it - but they can't win
Umm..actually, most observers have been noticing this for some time, certainly since 2006 when we first wrote on New Search - the reason the meme has taken off like wildfire now is that it has got noticeably worse since 2009! Anyway, the argument is that Gooogle is now putting in Niche Search un-noticed by all us dumbos, so even though its "not a battle they could win", they have in fact won:
But they have won anyway
So, all the current people who are seeing a problem are clearly wrong, its gone now - move on, nothing to see here. Unfortunately for this argument, SEO Watch did a survey at the same time and found that (surprise) there was a problem, Bing getting better results by 63 to 52 points.
There is the start of a good debate for New Search aficionados on Stowe Boyd's post here:
We will find everything through social relationships: what washing machine to buy, or the best Thai restaurant in Beacon NY, or the company that makes the horizontal corduroys. people that care about these issues, and to who we matter, will share meaning with us: they have beliefs that they can justify, also called knowledge.
Thursday, January 6. 2011
Interesting response on Hacker News to my and others' posts on Google Adspam:
Given the current high-ranking thread about spammy sites in Google results, it strikes me that a very simple solution would be to let logged-in users blacklist sites.
Comments are interesting too - firstly, there is the exploration of a social graph blacklisting:
I think a personal black-list would be ideal initially as those most motivated would be most helped, i.e. the majority of people who might not care about the status quo are then not impacted at all.
Other ideas are a "report spam" system:
I want a search results page similar to the "Priority Inbox" we got recently in gmail. Set sane defaults and let me override them with "Important/Notimportant" buttons (or thumbs up/down or whatever) next to results. Let it learn what I think is a good result for my needs. If you make it a little bit social, make sure you weight other people's opinions by how much they agree with my own in other areas (making it harder for sockpuppets to muddy the waters)
All in all it goes to a theme we have been banging on for quite a while about, ie allowing user-defined filtering for web-services (Whether Google, Twitter etc). But as many point out, Google is conflicted here, as these sites generate fast churn of click through pages so a high Ad turnover - so are probably not motivated to do it:
Wednesday, January 5. 2011
A million mailboxes were seemingly seeded with Quora invites over the last few days (grassroots or astroturf? the jury is still out, the question unanswered), a million flowers thenceforth bloomed, and our twitterstream filled up with irritating autotwts..
A quick aside on Crowdsourced Q&A services - they have been touted as "alternate search" for years, but crowdsourciing answers has its own problems. Their history is one of being useful at first, but dying under the weight of spurious questions, cr*p answers and pure spam over time. But it is a good time now, as those that launched before did it against a background of Google search being effective, but that is not so much the case anymore - so timing is propitious.
Here is the Broadstuff's take on its probable future therefore:
So the key task for the team is to build the system such that dumb punter impacts are mitigated, that spam is minimised, and that - above all - the service gives enough good answers to questions. Do this, and there is an alternative search system in the making, Fail to do do and it joins the ghost town question services out there, with their tumbleweeds of hanging, unanswered questions except for spam replies. Personally I have always felt that the way to do this is rank the answerers as well as the answers and allow user filtering.
However, I do fail to see the difference between these services and a million "vertical" system on Yahoo groups and various web services, with the exception that you only join one service here rather than all those verticals - but then you also have to deal with unfocussed answers and grockles (Having read quite a few threads, I cannot say that the answers are of higher quality than I have seen anywhere else, but at least at the moment there are answers).
So good luck....they will need it.
Update - Good Lord - a Techcrunch story critiqueing it a tad (TechCrunch has been promoting Quora something awful!)
Update - and now, 3 weeks later, a GigaOm story in the face of universal "meh" We salute thee, PR team of Quora, but you are turning it into a bit of case study!
Monday, February 15. 2010
Practice is starting to look like theory. When Social Networking started, it was clear that people would use it to find things that way as well as using search engines. Its just that the volumes were small and Social Networks, by and large, had crap comms to share things by. Twitter changed all that, and now Facebook - having implemented far better and simpler comms - is finding the same:
For what its worth this blog has had more referrals from Twitter than Google for some time, which has made us far less impressed with all the arcania of Google SEO).
But now that its happening in scale, it puts Google's model under pressure as it is simply cruder for Advertising:
But Social search just isn't scalable without the Signal to Noise ratio getting unmanageable - so we expect to see a whole raft of Social Algorithms emerge to help automate, speed up and filter the process. How do we know this - because we, and a whole lot of other people we know, are fiddling with them, and they show a lot of early promise.
Steve Rubel hypothesized Buzz was launched half baked and half cocked to get out there before Facebook launched its FBMail service, and that may be so - but to our mind the real Buzz is in the algorithms dealing with the blend of social graph and social data.
Sunday, February 7. 2010
Search Engine Usage by Adoption Type (Advertising Age)
Not only can one tell a lot about you by the searches you make, seems a lot can also be deduced from the search engine we use, as a WPP survey of 17,000 US internet users shows - Ad Age:
What does your search engine say about you? Well, if it's Bing, you're probably an early adopter, but you also visit, shop and ultimately make purchases from Walmart more than other search-engine users. Google searchers, on the other hand, are partial to Target and Amazon, and Yahoo searchers have a strong preference for wireless service from AT&T and Sprint.
Fascinating. I've used Dogpile for nigh on 10 years, it searches all the other engines and aggregates their result for you. I love the "parasite search engine" design (I wonder what that says about me ). Interesting that it hasn't really taken off though.
Update - realised there is a bigger story here. In an age where we see Google moving to mainstream advertising at the Superbowl (while Pepsi eschews it), clearly which search engine you use is becoming as much a branding battle as for any other commodity product - ie there is little inherent technical differentiation left. Increasingly search engines will try and identify demographics they want for Advertising purposes, and set out their brands to recruit them. Next step is diversofoed brands for different segments - Google Lite, Bing Premier etc - and celebrity sponsorship. Oh joy.....
Tuesday, January 19. 2010
I wondered how long it would take someone to build a system that obfuscates your data from Google. We looked at how to do it a few years ago, but the best short term default seemed to use Dogpile, a "parasite" search engine, or similar - yes, they have your data but its a far smaller operation. But that was just search. . However, a smart hacker has come up with a way of doing it. Forbes:
Now an independent security researcher who goes by the name Moxie Marlinspike is making Web users a counter-offer: Take Google's ( GOOG - news - people ) giveaways and keep your privacy too.
You can get the Firefox addition here. Marlinspike is offering up his code so you can build your own, so well worth a look methinks. I predict a large increase in Google services that have to be signed in for. I wonder if the Open ID people may have looked at this, but they have a different gig right now - but maybe we have a bigger need for Private rather than Open ID.
Wednesday, January 13. 2010
You all know by now that Google is pulling out of China (actually, its "May pull out if non censorship request is not met", but that is very unlikely). The reason given is a massive attack on the accounts of Chinese dissidents in Google (and other) companies' servers. But, as was pointed out on the BBC Today program this morning, everyone told Google that would happen when they went in 4 years ago. The Google response this morning was that they hoped that by engaging, Chinese behaviour would change
Another reading - the RealEkonomik reading - is that Google knew this would happen, but that it could not afford to ignore such a huge emerging market so went in with eyes wide open, albeit possibly with rose tinted glasses. As Shefaly Yogendra points out, strategy in China is a new Great Game.
So what's changed? We suspect that as well as an increase in attacks on the servers (Why is China doing it only a problem now, though?*), three other factors have driven them:
- Their reputation in the far more valuable US and European Union market is diving, and both areas are looking at regulation to curb their activity.
Let's be clear - a publicly owned corporation does not do things on points of ideological principal, Google is doing this as it believes it serves its own best economic interests here and now. In other words, in the short term it would hopefully do Google a lot of PR good (and allay some legal activity) to give China short shrift. In the long term this may prove to be short sighted, but its a reversible decision and in the long term we are all dead - and the current management will long since have taken their pension pots into retirement. Besides, doors are still open....
All that said, it is still thrilling - regardless of the reasons why, they are doing the ethical thing.... Interesting Times are surely ahead.
* It will be interesting to see if other companies like Microsoft and Yahoo report increasing attacks. Also, surely this is a government to government issue if as widespread as claimed? (Update to this - a statement from Hilary Clinton)
Friday, January 1. 2010
Techmeme comes up with their top 10 stories of the year - and 7 out of 10 are Apple or Google stories. The other 3 are TechCrunch (with an Apple substory), Friendfeed (ex Googlers) and eBay/Skype. How was such a monochrome palette selected - sez TM:
Hmmm... to an extent what you get out of it depends on what you put into it. I've noticed that TM is biassing more and more towards mainstream sources, so maybe that is an explanation.
But even so - seriously - no Twitter, Hulu etc? Or the AT&T/4Chan and PirateBay stories that hit top spots on Digg? If I look at the traffic on our blog, we covered all the above stories and they were not the biggest runners (see here). Be interesting to see what the top runners on TechCrunch, Mashable etc. were.
One wonders about the exact selection criteria! (Update - Techmeme's Gabe Rivera has commented below, explaining its based on link count. That makes more sense as I suspect Google and Apple have more fanboi 2.0's than say Microsoft, but also - in my humble opinion - shows the risks of pure link based ratings)
Now I really like TechMeme, and am a great fan of its model, I've always seen them as a leading example of how media may work in the future - but the risk they run here is becoming Yet Another Mainstream Tech News Organ..... so why?
Does becoming a me-too storyboard system create sufficient value from the (hypothesized) increased volumes? One also therefore starts to wonder more deeply about how they are funded, and the implications thereof!
More Broad Stuff
Poll of the Week
Will Augmented reality just be a flash in the pan?
Creative Commons Licence
Original content in this work is licensed under a Creative Commons License