Web Services / Cloud Computing

Personal, Private, Portable - and baked into the infrastructure... not a Pi in the (Cloudy) Sky idea anymore

Worries about how private data in the cloud is has set entrepreneurial minds a-spinning - TechCrunch:

It’s clear that Snowden has shone a light on something very rotten lurking at the core of mainstream cloud services. Sure, web mail clients are handy — and you get a chunk of free storage — but you’re also effectively handing the keys to your inbox over to the NSA (et al) should they happen to feel like dipping into your personal correspondence one lunchtime or other.

And of course it’s a given that your personal data will be mined by the service provider; it’s a free service, so your privacy is the product. This general snooping on/data-mining of users is a routine business model for the majority of free cloud-based services and apps.

Multiple big name tech giants have built hugely lucrative businesses off of selling your likes and dislikes to advertisers in exchange for furnishing you with a platform where you can share that data in the first place. Or store the personal stuff they need to peek at to gather sellable intel on you.

Plus there’s also the sloppy security of (certain) startups that leak your personal info all over the Internets just because they’re more interested in expanding their user-base than expending effort on adequate security measures.

The article is about a hardware based private cloud system startup - a personal, encrypted cloud device that plugs into your home Internet via Ethernet and provides cloud storage for your digital content without having to go through any third party intermediaries, and with support for unlimited expansion of said storage — either by chaining together multiple Pixeoms, or by plugging in additional USB hard drives.

I got quite excited about this. Private Personal Clouds are not that new, but Portable ones are much more interesting. Just as big servers in the office was a passing through phase, big servers in the cloud will be too, and at some point where Moore's Law trumped Zuckerberg's Law, the Portable Private Personal Cloud was bound to appear. In fact we even did a high level design* for one for some other people a year or so ago, and at Broadstuff Towers Mr Short has been known to get very excited about getting his hands on Raspberry Pi for this sort off thing (and getting his hands on the little computer Pi as well...) so i guess there will be more of these devices to come.

And of course, we can't end this post without the inevitable reference to an Internet (full) Of (these) Things, and what that will mean for Fridges everywhere**

*The next step up from the paper serviette doodle.

**More Pi.

Marc Andreessen thinks that the clock is ticking on Oracle and other old-line software and infrastructure companies - Business Insider.

His evidence: not a single one of Andreessen-Horowitz's startup investments use Oracle software. They all use cloud-based alternatives instead.

............

"Ten years ago, it was a joke: you'd raise $20 million in venture capital and write a $4 or $5 million check to Oracle, Sun, BEA, and EMC....When it started, Salesforce looked like a toy compared with Siebel. Look ahead five years later, it's obviously better. Not a single one of our startups uses Oracle."

The thing was that Oracle, BEA, EMC - and even Sun - were not appropriate for startups even 10 years ago, and everyone knew it - but you had to "look big fast" in the high hype world of dotcom share price pimping - and every dotcom with attitude was going to be huge in 2 years, of course - so get the Big Boy systems in now rather than buy a decent SME accounts package now and buy Oracle later when you were bigger. Brave indeed was the dotcom startup who went for the appropriate technology, therefore . What Marc also doesn't mention is that many times these large companies would give you the kit on some sort of for-shares or low interest long life loan basis, so you would be financially daft not to take it.

My main issue with Oracle (and the other big players by and large) is that they have done little in the last 10 years to redress that known weakness - and you would have thought Sun's demise would be a lesson. They saw Hosted solutions coming 10 years ago, so this cannot be a surprise - and thus why no action?

The reason is the fairly standard large company issue - addiction to the current business models, and finding it very hard to invest in new technologies that don't "move the needle" and are unreliable by "tried and trusted" standards. In fact, Oracle innovation has typically been "buying the market" as its innovation ploy - either buying out sector competitors, or small companies with interesting products - and then sadly (and probably even unintentionally) slowly strangling them by underinvesting and burying them in corporate treacle.

(This was part of Marc's argument that there Is No Bubble- he believes the cloud computing revolution has made the titanic valuations of major Web 2.0 companies like Facebook and Twitter completely justified - which we just don't buy)

So my prediction is slightly different to Marc's - I would predict that as the Cloud market grows, and becomes more stable, then Oracle, SAP, IBM et al will increasingly just buy their way in. Give it 3 years or so, and Marc's investments will be using Oracle

And as totitanic valuations being justified, we shall see.....

JoltID is the company set up by the founders that has the Skype P2P IP in it, that eBay failed to buy and thus scuppered their attempts to sell Skype easily. One assumes Microsoft bought the IP but I cannot see it in print anywhere (and having seen "Deal Fever" at first hand I can easily believe they didn't....).

Anyway, if they did buy it then my view is that Microsoft overpaid, but it's a rational strategic move so good luck - they will need it, the devil to creating value now is in the execution into the SOHO/SME business market - and execution on acquired companies is not a Microsoft strong suit.

If they didn't, then my view is that they are complete dingbats.....

There is a lot of Sturm und Drang about Amazon's EC2 "Cloud" system collapse over the last 2 days. the main thing it shows me is that most people fundamentally misunderstand networked system reliability. Allow me to make a quick calculation:

- there are c 356 days in a year, 24 hours a day, that is 8544 hours in a year
- a "Five Nines" reliable system like a voice telephone network, with massive investment and decades of operating experience will be up 99.999%, ie you need to expect about 0.09 hours or c 5 minutes of total system failure a year.
- Cloud systems are nowhere near Five Nines reliable, even Broadband isn't there. In my 10 years experience with web hosting/ASPs I'd say that 99.9% reliability would be a great target to aim for still, ie you need to assume c 8 - 9 hours of total failure every year.
- You can back this up by parallelling systems, but to be truly redundant this needs to be in a different datacentre, preferably in a different country, ideally with a different operator so there is no problem bleed and error buildup. In such a perfect case, you can assume you will get two systems of 99.9% reliability ensuring one of them is up 99.99% of the time, ie only one hour a year when both have crashed at the same time.

Also, the sorts of datacentres operating consistently (ie are prepared to sign a service level agreement - SLA - with penalties) at even "Three Nines" are pretty high end, your average cash starved startup sitting on a standrad deal on one cluster of a virtual shared hosting service like Amazon (as far as I recall they don't have an SLA, just Best Endeavours to hit 99.95%*) is probably not going to see even that level of reliability. Assuming they see something like 99.5% reliability, that means you must assume c 2 days of outage a year. Even a fully redundant system like I have described above, at 99.5% will give about 1 days complete outage a year.

In other words, these swans are going to be Black, and they are entirely predictable.

I'm afraid the Cloud industry tends to oversell it's capability for reliability somewhat (what's .499% between friends - well, about 2 days outage a year actually) , you will know its got to Telkco utility levels of reliability when it's provided by massive telecom utility type providers, not run off the back of companies' whose main business is doing something else. If you ever read Nick Carr's book The Big Switch you will see we are in the equivalent of where the 1920's was for electricity, where the industry is moving from fairly unreliable, non standardised services, run by people selling off spare capacity, on to dedicated service providers. No reason not to have Cloud now, (just as no reason not to have electricity in 1920) but you wouldn't bet an entire livelihood (or the farm) on it.

The one thing that Amazon needs to learn though, is telling customers what is going on early and honestly - survey after survey over the last 20 years shows that customers are much more confident, and ultimately happier with you if you tell them - even if it is just that something is wrong - and put in place a comms update plan. Amazon's Cloud apostles dropped off the Twitter radar for hours as this happened, and the initially company clammed up, except for rather legalese communications it seems. No datacentre operator worth their salt wouldn't have system healthcheckers in place so that at least they could tell customers where faults were very quickly, and what their disaster recovery procedures were going to be.

(For the record - we have an Amazon instance, but we don't host the blog or website on it. Our advice remains to use the cloud by all means but also to have your own systems running in hot backup and able to do short term switchover and be able to run the system at normal mean loadings - and ensure any data that you need if it goes down is mirrored)

* I was wrong, one can apply for credits now - but it's money off the rental cost, not business damage.

Update - and here come the Cloud snow jobs - this one is just too (sadly) funny for words but fortunately it's petard is well hoisted on Slshdot - it's on O'Reilly, which does surprise me.

Fifth up on Wednesday at the Financial Times Digital Media & Broadcasting conference was "The Real time Challenges of Media" with a panel discussion featuring:

- Marc Landsberg, Chief Executive Officer, MRM Worldwide
- Ian Rosarius, Managing Director- TV and Content, BT
- Kevin Walsh, Chief Technology Officer, Oracle Asia Research And Development Centers

One of the more realistic discussions of The Cloud that I have heard in a conference (probably because it wasn't a Cloud conference) - some notes:

- Cloud is an "Old New Thing" - it has failed before (think thin Client, Network Computer) for the same reasons it may still fail now
- It's still far harder than anyone thinks to orchestarte processes across multiple cloud platforms
- Main benefit is not OPEX economics but CAPEX - you don't have to build to peak load [Not tortally true in my experience, Amazon still wants us to predict our unpredictable peaks]
- There is as yet no demonstrable "consumer utility" over existing "good enoughs"
- However, the Cloud is forcing rational industry behavour by driving adoption of de jure and de facto standards
- Panel felt the Cliud's main role will be to enhance the capability of the relatively "dumber" mobile devices - smartphones, tablets - than the PC/laptop ecosystem

On the evolution of Web TV, which is a specific instance of The Cloud

- TVs and other simple black box devices will take over the role currently done by complex devices (STBs, PCs)
- However it is still very early days with standards, how EPGs work etc - it is like TV in the 1950's
- What will win is the ability to provide a seamless experience - the techniology will exist long before a decent experience (Chasm description)
- There will be a period of consumer confusion, that smarter players will use to gain consumer confidence
- There is a major battle emerging about who owns the (user) data in the value chain - everyone wants it
- There will probably have to be far more work done on people's privacy, and who conyrols the data - trusted providers may well have a market benefit

The irony is that history tells you that apple are the past masters at creating the first Seamless Exoerience in media vale chain after value chain (PC, Music, Smartphone, tablet...DTV next?)

Last discussion was on good old Net Neutrailty, the point was made very clearly by BT that the current UK backbone was sized in the early noughties for low end DSL speeds not broadcast scale High Definition video to a nation, the investment to upgrade to mass big bandwidth now will be huge, and the consumer cannot bear the entire cost. Content providers who use up massive amounts of the network capacity and want real time transport will have to pay more than users with small bandwitdh, low real time synch needs [As has happened with every other utility in history].

Loved the "Wisdom of the Clouds" title, it's pinched from Paul Carr's piece on the issues that the Wikileaks affaire brought up - or rather, the way Cloud players rolled over when Sen Liebermann roughed them up and couldn't hand over user data fast enough (Twitter being the honourable exception).

It used to be that if the US government wanted access to documents or letters in my possession they’d have to subpoena me directly. As a foreign citizen there are all sorts of ways I could fight the request – and it was at least my choice whether to do so. As someone living in the US I also had the whole weight of the 4th Amendment on my side. Now, with everything in the cloud, the decision whether to hand over my personal information is almost entirely out of my hands. And unless, as happened with Twitter, the company storing my data decides to fight for openness on my behalf, there’s every possibility that I won’t even hear about the request until it’s too late. That’s just not how things should work in a free society.

We agree totally (see here) and its no accident that many of those most interested in telling YOU to open up run very closed shops themselves. Look at what they do, not what they say!

We would also argue that there is another, older reason to question wisdom of Clouds, and this is simply what happens if it goes away - all that data - gone. There are 3 aspects:

(i) Cloud not accessable from current location - this means potentually a whole lot of work you need to do can't be done. I am amazed at the increasing number of people who look at me as if I'm nuts when I say i use Thunderbird to actually store all my emails on my own hard drive(s), not just for backup but to work when connection is unavailable. It beggars belief that there are people who can't access their email locally, but there you go... the madding crowd and the Cloud and all that.

(ii) Cloud connection/service temporarily down - work stopper, especially if it is core data. Why would anyone take this risk. Clouds cannot be 99.99% reliable, and Murphy's Law clearly sates that the .001% will be at the most inconvenient time possible. No sane CIO would not have a backup strategy

(iii) Cloud service disappears - either it kocks you off (cf Wikileaks) or goes pop. In either case good luck getting your stuff back. At least with a hard disk crash you can try and sort it out yourself.

We have been worried for quite soem time about privacy implications of Cloud storage, especially on free services - though as the Wikileaks affaire proves. its not just free services that can play fast and loose.

Long term readers of this blog will know our recommendation is to go for hybrid services - use the cloud by all means , but keep a hot running local backup. This is borne by our experience over 20 years dealing with networked and hosted services. You cannot engineer these thinsg to be 100% reliable even at infinite cost. at affordable cost they will be a darn sight less fr quite a whiel. (I note the Cloud lobby can no longer claim that Cloud services are more economical)

Another thing we have been looking at is where can you put secure servers, ie ones that can't be pried into by powerful bodies. which countries would host such a service? (Encryption is not enough, as if the host country wants the data badly enough they can get it, and its no guarantee that your encryption can't be broken (or more likely, compromised by man-in-the-middle attacks) if the hoster is in cahoots with said snooping entity.

Salon puts it very well....

The WikiLeaks affair is highlighting the Internet's soft underbelly: the intermediaries on which we all rely to store our information and make it available. We are learning, to our dismay, that we cannot trust them. Combine that with increasing government intervention, we're also learning that the Internet is somewhat easier to censor than we'd assumed.

This should worry anyone who believes that we're going to move our data and online lives into the fabled "cloud" -- the diffused online array of hardware and services where, proponents say, we can do our online work, play and commerce without the need for storing data on our own personal computers. Trusting the cloud is becoming an act of faith, and it's time to question that faith.

And the situation should absolutely chill everyone who believes in free speech -- and especially the people who call themselves journalists. Sadly, however, too many of them have been cheering on people who want to make WikiLeaks disappear. Do they realize that it could be their own turn someday?

Given that the economic case for Cloud Computing seems to be floating ever further away, and its most punted current rationale is "risk reduction", Wikileaks shows The Cloud's promise of risk reduction is whispy at best and that there is only one way to reduce risk and that is have your own servers, preferably not in the US. And ideally have your own Internet as well....

The correct retort to any Cloud Apostle right now is probably somewhere between a snort of derision and a well placed boot on the seat of the trousers....

John Battelle on the "Next Big Thing":

Today I was in a meeting with a number of consultants to a very large technology company. Their job: market research, essentially. They called to ask me my thoughts on the media and technology world, in particular as it might play out in the next five or so years. They were responsible for helping the Fortune 50 company navigate an increasingly complicated world.

.............

I've been a student of technology cycles for a couple of decades, and often times what's directly in front of you is, in fact, the next big thing.

So when I got this question: "What's the next big thing after social?" I didn't lose a beat in answering: "Location."

Now, many, many folks before me have been saying this for years. I'm in no way first. But I'm an early convert, in particular, as it relates to what I call the conversation economy. And the reason is simple: Once someone can declare where they are, they add extraordinary context to both search and social, and to their expectations of what a search or a social connection might yield. For an example, see The Gap Scenario.

In short, location is a key factor in the future of search, social, commerce, and media, among a lot of other things. And that's why the news today that Google's Marissa Mayer, long the VP of Search Products at Google, is taking over responsibilities for the location business, strikes me as a Big Deal.

That Google is re-allocating Marissa Mayer here is a sign of a hyper market, to be sure. It is a time for image, as with location there is still very little substance - generation 1 failed, generation 2 are resorting to ever gamier tricks to keep user interest. But to be very publicly Seen To Be Doing Something.....well, just look at the headlines generated today to start!

The holy grail of location is localised, personalised, highly valuable advertising (compared to the appallingly low rates that most social advertising gets)

Well, as readers of this blog will know, we think Location based service opportunities are overhyped (that has never stopped fools from being parted with their money, of course) for the following reasons:

1. Fundamentally, its a feature, not a stand-alone business. Location will be integrated into other things you do (cf Twitter). This means, apart from a few Location companies being bought at fairly low cost, we are not going to see large exits. The real money will be made by people like Google rolling it out into their large distribution channel.

2. Dreams of high advertising values are a chimera - the mass audience volumes aren't there, and the personalisation/understanding of intent is lacking unless there is a deep understanding of the rest of the customer dataset. This will be like Second Life for a few months as every Big Brand feels it has to be seen, and then it will fall away as they realise that you don't want to buy X every time you go near place Y. And the privacy safeguards required - certainly by EU law - will limit a lot of the data rapine that will be required hor high personalisation.

3. Most of the "location based" services that add a lot of value to the user do not lend themselves to advertising anyway, and also they tend to require other services to interact with them (see point 1 above) which dilutes/complicates/adds cost to the Ad value chain. Can you imagine if every time you wanted to read a map on your iPhone you got an Ad pre-roll? Or that you had to syndicate your personal data across 4 service providers every time you needed to know the nearest X joint, and they had to bump up the Ad rates to you to cover the cost of paying 4 separate people and managing the process?

In oher words a small number of small companies will sell at a nice little price, the big guys will build it in as a "stickiness must have" and it will be just another feature in the service mix. Location will be like email - a feature you must offer, but not something you can intrude on too much before people shy away from it. As to the Googleshuffle, whisper it who dares, but Google Search is in need of a major revamp that probably only a fresh head can bring, so that is probably another contributing factor.

As to what the Next Big Thing really is - well, thats obvious, you just look at where the money is* - it's Web TV 2.0. But more on that later......

*Bear in mind that Social Media is still a zero billion dollar industry - there is as yet no evidence that Facebook et al can turn a profit. In theory they can, but whether it is anything like what is required to justify their valuation requires you to believe some far fetched hypotheses, frankly. The only real money made to date has been by large companies paying hype-level prices for social media companies at the top of it's hype curve. Which, of course, explains the Rush for Location

Had to install iTunes to set up an iPhone for a friend the other day, finished it all nearly 100Mb of bloatware and 1 hour later. WTF sez I, what could possibly justify such a huge program just for that? Well, I stumbled upon this Xconomy post about the Why:

If you’re not convinced about iTunes’ cruftiness, let me take you on a tour of the program’s main functions. This is a long list, but bear with me:

• It lets you rip CDs to digital formats and play the new files

• It lets you burn new CDs from your digital files

• It lets you print jewel-case inserts for your newly burned CDs

• It gives you several ways of visualizing your media collection, including Cover Flow

• It lets you curate your music collection with ratings and the like

• It lets you create playlists from subsets of your music collection

• Its “Genius” feature can automatically create new playlists based on your listening habits

• It includes a music equalizer and other sound processing features

• It stores copies of your purchased albums, TV shows, and movies

• It stores copies of your downloaded podcasts and iTunes U videos

• It stores copies of the iBooks editions, PDFs, and audiobooks that you may be consuming on your iPhone or iPad

• It stores copies of all of your iPhone and iPad apps

• The Genius function can suggest apps you might like based on your past downloads

• It stores copies of your iPhone ringtones (but it doesn’t let you make your own ringtones anymore)

• It connects to hundreds of streaming Internet radio stations

• It is the leading podcasting client, automatically downloading new audio and video podcasts to which you have subscribed

• It is the gateway to the iTunes Store, which is really seven separate stores for music, movies, TV shows, apps, podcasts, audio books, and university lectures

• It’s the only way to access the new Ping social network

• It’s the hub for sharing music across your home wireless network

• If you have a new iPhone or iPad, you have to use iTunes to activate cellular or data plans

• It synchronizes the music, movies, or TV shows that you buy on your computer to your iPod, iPhone, or iPad, and vice versa

• It can transcode video in certain PC formats such as QuickTime into formats that are playable on iPods, iPhones, iPads, and Apple TV

• It synchronizes your iCal calendar with the calendars on your iPod, iPhone, or iPad; it also synchronizes your address books and any content in your Notes app

• It is the conduit for installing the MobileMe control panel, if you want to synchronize data automatically across your PC and your Apple devices

• It stores voice memos recorded using the iPhone’s built-in voice memo app

• It’s the repository for music and video files embedded in documents created using Apple’s iWork and iLife productivity applications

• It interacts with the Remote app, which lets you control your media collection from an iPhone, iPod Touch, or iPad

Any program that can print jewel case inserts and share my music preferences with my friends is starting to sound a lot like that giant clot of bubble gum.

As Tim O'Reilly notes, iTunes bloatware may be the undoing of the Apple bid for Global Domination. And it starts insidiously - I wish I'd read Jas Dhaliwal's post on how to avoid megabloatage first:

Many people do not want to install the other applications. However, Apple does not allow the option to install individual components. The good news, is that it very easy just to install the individual components that you want.

Firstly, you need to install Winrar. This application will allow you to ‘unpack’ your downloaded iTunesSetup.exe file. Locate your file, and right click on it. You will be presented with a number of menu options, select ‘Extract Here’.

(Jas's post has pictures n' all, so go there if you want to unbloat your iTunes)

Winrar will now extract the iTunes installation package and will reveal the individual MSI setup files. You can now double click on the iTunes.msi file to install iTunes without the other applications.

Please excuse me now - me and iTunes are going to have a little tete a tete about slimming......

The New York Times on Location based services:

Venture capitalists have poured $115 million into location start-ups since last year, according to the National Venture Capital Association, and companies like Starbucks and Gap have offered special deals to users of such services who visited their stores.

But for all the attention and money these apps and Web sites are getting, adoption has so far been largely confined to pockets of young, technically adept urbanites. Just 4 percent of Americans have tried location-based services, and 1 percent use them weekly, according to Forrester Research. Eighty percent of those who have tried them are men, and 70 percent are between 19 and 35.

The Fist Generation services (Loopt, Dopplr etc) have by and lerge failed to gain traction. So far what has worked with 2nd generation location based services is game based rewards (eg becoming mayor of a place) but these rewards are ultimately about bribery (eg becoming mayor gets you a free offer). Shopkick plays for the endgame by giving you money off coupons when you shop at a certain shop - it's the online equivalent of newspaper coupon clipouts.

But never fear, the CEO of Loopt claims that people born after 1981 (ie below 30) have lower privacy requirements, and thus a larger demographic will emerge year by year:

“The magic age is people born after 1981," said Sam Altman in a New York Times article. “That’s the cut-off for us where we see a big change in privacy settings and user acceptance.”

So, two contradictory views - the NYT arging that the market is limited, the CEO of Loopt arguing that the only way is up. I have another explanation for the people after 1981 being less privacy aware, and this is simply that they are young and have less to lose - as the NYT notes:

Stephanie Angelucci, who is 30 and lives in North Beach, Md., updates her MySpace page with photos of her babies, news about her health and testaments to her love of sailing. But she won’t use location apps.

“I don’t like broadcasting where we are or when my husband’s gone, just for safety reasons,” she said. And privacy concerns aside, she doesn’t see the need: “We go to playtime, the park and the grocery store. My life isn’t exciting enough to broadcast where I am and what I do.”

In other words, perhaps at (about) 30 people start to have responsibilities, and for various reasons become far less interested in displaying their location.