Disruptive Telephony

For the first time in 3 years, Skype was down today - and as I write this is still in the process of slowly coming back online. A ton of articles were written today, mostly all pointing back to Skype's blog post or status update, which most importantly said this (I've shortened it a bit):

Some of these computers are what we call ‘supernodes’ – they act a bit like phone directories for Skype. If you want to talk to someone, and your Skype app can’t find them immediately ... your computer or phone will first try to find a supernode to figure out how to reach them.

Under normal circumstances, there are a large number of supernodes available. Unfortunately, today, many of them were taken offline by a problem affecting some versions of Skype. As Skype relies on being able to maintain contact with supernodes, it may appear offline for some of you.

Let's explain this a bit more.

Explaining Supernodes

If you go back and read my primer on the technology behind Skype and P2P networks, I described supernodes as Skype clients that are on the public Internet and NOT behind a firewall or NAT device that broker the communication between two Skype clients. In a very simplistic view, the picture looks like this:

As I note in the update section to that post, the Skype clients acting as "supernodes":

perform the somewhat limited functions of connecting nodes together, providing a distributed database and choosing appropriate nodes to act as "relay nodes" when necessary.

The supernodes are what connect invidividual Skype clients to each other and create the P2P "overlay network"... the "cloud"... that connects all Skype clients to each other.

These "supernodes" run the regular Skype software. The ONLY difference is that they are on the public Internet. So if you are running Skype on a computer - and you are NOT behind a firewall, there is a chance that your computer could become a supernode. That's just how Skype works. So there are a lot of these supernodes out on the public Internet:

Here's the thing... EVERY Skype client is connected out to a supernode. You have to be, in order to be connected to the larger directory of Skype users and for them to know how to reach you. (Note that Skype clients behind the same firewall may not be connected to the same supernode.) So it may look like this:

The supernodes are then connected to each other... creating Skype's globally distributed directory database, which in a simplified form you could think of like this:

(Skype's supernode connection algorithm is presumably more complex than the simple mesh I'm showing here... but the point is that they are connected to each other.)

Now, Skype's picture is not exactly like this. We know from the explanations of the 2007 outage that Skype uses a hybrid architecture that involves some "authentication servers" that Skype clients connect to in order to first be granted access to the Skype P2P cloud. I'm not aware of anyone publishing technical details on exactly how those authentication servers connect into the Skype infrastructure, but let's just say it looks something like this:

Skype clients need to connect to these authentication servers in order to validate their username and password, and presumably to validate their calling plan, how much money they have left in their account for calls, etc.

Now, the cool part about the "self-healing" aspect of the supernode architecture is that if a supernode goes down, Skype clients will simply attach to another supernode:

The problem with the outage today seems to be, from Skype's explanation, that a great number of supernodes went offline, tearing apart the fabric of Skype's P2P network overlay:

OOPS.

Something broke. We don't know what. Skype's blog post says only:

Unfortunately, today, many of them were taken offline by a problem affecting some versions of Skype.

What was the "problem affecting some versions of Skype"? No clue. Was it a software update that somehow affected the supernode algorithm? Did it affect the communication with clients?

No clue.

But according to Skype, that's what happened. Hopefully they will be a bit more forthcoming soon (although perhaps NOT, given their pre-IPO status), but at the moment that's all we have to go on.

My guess would be that there might also have been "cascading failures" in this scenario. If there was, say, a software update affecting some supernodes, as those supernodes dropped offline, the increased load of Skype clients trying to connect to online supernodes might have caused some of them to then drop offline. Or when a supernode came back online, it may have been overwhelmed by the quantity of connection requests and soon failed again. As I said, that's purely a guess... but you could see those kind of failures happening in a situation like this.

Skype's "Solution"

As a solution, Skype's blog post says this:

What are we doing to help? Our engineers are creating new ‘mega-supernodes’ as fast as they can, which should gradually return things to normal. This may take a few hours, and we sincerely apologise for the disruption to your conversations. Some features, like group video calling, may take longer to return to normal.

No details yet on what these "mega-supernodes" are, but some speculation is that instead of relying on individual Skype client computers to "become" supernodes, Skype is going out and setting up computers/servers specifically as supernodes. Rather than rely on potentially unstable computers, Skype goes out and gets some rock solid servers under their own control and sets those up as supernodes.

Maybe that's what a "mega-supernode" is. Maybe it's a higher level supernode... to which "regular" supernodes connect. Again under Skype's control... but providing a tighter core P2P network that houses the overall directly.

We don't know yet... but those are the kind of things Skype could be doing. Again, hopefully we'll get more details soon... although we'll have to see.

As I write this, my Skype client shows 4.5 million users online... it's the beginning of the day in Europe and I'm sure folks there are trying to get online. Hopefully Skype will be getting their network back online soon.

And hopefully we'll get some better technical explanations, too!

---

NOTE #1: It should be noted that there are other types of "servers" connected to the Skype P2P cloud beyond the authentication servers. There are also the servers and gateways used for SkypeOut and SkypeIn, gateways to mobile operators, web presence servers, etc. I left them out for the simplicity of the drawing.

NOTE #2: I am not an employee of Skype and do not have any inside information about the workings of Skype. The information in this article is based on what technical material Skype has made publicly available plus information a number of us have been gathering over the years. It may or may not be accurate.

---

If you found this post interesting or useful, please consider either:

• subscribing to my email newsletter;
• following me on Twitter; or
• subscribing to the RSS feed

---

<shameless self-promotion>

With Christmas fast approaching, are you looking for a last-minute gift for someone you know working with telecommunications or security?

If so, may I suggest a book written by a certain someone called, oh, Seven Deadliest Unified Communications Attacks? You can order it from sites like Amazon.com and have the book delivered this week before Christmas!

The book will help whomever you give it to understand what the real threats to communications networks are today - and also what the real solutions are. Here's a video I made to explain why I wrote the book:

</shameless self-promotion> :-)

---

If you found this post interesting or useful, please consider either:

• subscribing to my email newsletter;
• following me on Twitter; or
• subscribing to the RSS feed

---

It has happened twice this morning. People calling my phone number looking to talk to someone at Skype. For quite some time now (months), I have received occasional random phone calls from people looking for Skype and when I've asked it was usually because they searched in Google and ultimately somehow came up with my number (which I publish freely here on my blogs).

This morning, though, I asked the two gentlemen who called how they got my number, and these were there responses:

"I was calling MasterCard to report a fraud and they gave me this number for Skype."

"Capital One told me to call this number." (Capital One is a US credit card issuer.)

Oops.

You see, here is the fundamental problem:

SKYPE DOES NOT LIST A SINGLE PHONE NUMBER ON THEIR WEB SITE!

Don't believe me? Visit www.skype.com and try it yourself. Find a regular PSTN phone number... I dare you to try! (And if you do, please leave a comment here!)

The closest you may get is to the "Where is Skype?" page that lists Skype's Luxembourg address and an email address, but no phone number. Their press pages have a contact form, but no phone number. (Even Skype's news releases don't have contact info.)

This is not a new issue. People have been complaining about it in the forums for years (example 1, example 2). Tom Keating wrote about this issue back in 2005 on his blog - with lots of comments from people. And today people are filling up the comments on third-party sites like ContactHelp looking for a contact number.

And apparently some people are calling me.

I've asked people in the past and it has always seemed that: 1) because I write about Skype and show up in Google search results; and 2) because I list a phone number on my blog site... because of that people in their desperation call me to reach Skype.

Now it seems some of the largest US credit card companies are helping with that. (Probably because someone there went the search route, found my number, and entered it into some database.)

So now the question is ...

what should I do?

Should I start taking the calls and invoicing Skype? How could I monetize this? Or have a bit of fun? ;-)

The good news for me is that the number they are calling is my Google Voice number and so if the calls start to come frequently (there haven't been many... but 3 now this week), I can just redirect it into an IVR application that can redirect callers looking for Skype to Skype's email address. (Gee, I know a great platform on which to write such an app :-)

At the moment I'm more amused than annoyed. Skype's a rather large company these days and it's amusing to me that they would make it so difficult for customers to interact with them that those customers wind up using the web and ultimately reaching out to little old me sitting up here in New Hampshire in my home office.

Now ask me that in a week or two if MasterCard keeps sending calls my way... :-)

Oh, wait... there's my phone... should I answer it "Thank you for calling Skype"?

P.S. And, oh, Skype... if you don't want to staff up a call center to handle customer phone inquiries, there's some really amazing technology out there that let's you have a phone number people can call you on and you can give them "self-service" options. You know, those "IVR" thingies... with many options. Lots of companies offer this technology, including, oh, my employer, Voxeo. We even let you build those self-service portals across multiple channels, like voice, SMS, IM and Twitter- and we give you all sorts of cool analytics and other integration. In fact, we even let people call into our apps using Skype, so you could set up a self-service app that could be reached from either the PSTN or Skype... and we're all SIP at the backend so we could interconnect with your backend as well. We'd be happy to talk to you about it... OUR phone number is prominently displayed on our web site... and you also know how to reach me on Skype ;-)

P.P.S. It has been pointed out to me that Skype does offer support via live chat and email if you login to their website and visit https://support.skype.com/support_selection - No phone support, though.

Flickr credit: Phil Sexton

---

If you found this post interesting or useful, please consider either:

• subscribing to my email newsletter;
• following me on Twitter; or
• subscribing to the RSS feed

---

If you recall, back in July Skype announced that it was leasing a gigantic 90,000 square feet of space in Palo Alto. Now it appears that they have moved into the space and Skype's Jason Fischl posted some photos of the new digs to his Flickr account:

Looks like a fun office space to work in!

UPDATE: In an amusing bit of synchronicity, no sooner had I published this post then I saw in my Twitter feed that Skype's Peter Parkes just published a similar post (only with the full photos). Too funny...

---

If you found this post interesting or useful, please consider either:

• subscribing to my email newsletter;
• following me on Twitter; or
• subscribing to the RSS feed

---

Very cool to see that the Android 2.3 release includes a SIP stack:

The platform now includes a SIP protocol stack and framework API that lets developers build internet telephony applications. Using the API, applications can offer voice calling features without having to manage sessions, transport-level communication, or audio — these are handled transparently by the platform's SIP API and services.

The SIP API is available in the android.net.sip package. The key class is SipManager, which applications use to set up and manage SIP profiles, then initiate audio calls and receive audio calls. Once an audio call is established, applications can mute calls, turn on speaker mode, send DTMF tones, and more. Applications can also use the SipManager to create generic SIP connections.

Naturally this SIP stack is only available if the carrier and manufacturer allow it:

The platform’s underlying SIP stack and services are available on devices at the discretion of the manufacturer and associated carrier. For this reason, applications should use the isApiSupported() method to check whether SIP support is available, before exposing calling functionality to users.

Call me cynical, but I could see a number of carriers NOT allowing the SIP stack.

The Android team has also very helpfully provided a SIP demo application.

I also am intrigued by the "Near Field Communications" addition (if you don't know what NFC is, the Wikipedia entry is a good start). Looking forward to seeing what people do with that!

All in all Android 2.3 looks like a decent evolution of the platform... I'm definitely interested to see what people do with with SIP / VoIP capability. If you are an Android developer working with communications, what are you planning to do with it?

---

If you found this post interesting or useful, please consider either:

• subscribing to my email newsletter;
• following me on Twitter; or
• subscribing to the RSS feed

---

Skype today rolled out a very cool initiative with the UN High Commissioner for Refugees (UNHCR) where a new "low-bandwidth version" has been made available to UNHCR field staff so that they can communicate at no cost with family and friends. My immediate question on reading about it was:

Wouldn't many enterprises want this capability? Or a similar version?

The UNHCR Partnership

More on that in a moment, but first this is a great example to me of where VoIP can be used to enable communication in very difficult locations. In this case, the UNHCR already has Internet connectivity going into its locations through various means - this now allows the staff to use that connectivity for real-time communications. Consider where the UNHCR is using this software:

The new software has been tested successfully in Iraq, Sudan and Afghanistan and is now available to 1,010 staff stationed in remote locations in Afghanistan, Algeria, Bangladesh, Chad, Congo, Iraq, Kenya, Kyrgyzstan, Nepal, Somalia, Sri Lanka, Sudan, and Uganda.

By the end of 2010, we plan for that to grow to more than 2,072 members of staff across 60 UNHCR locations including Kenya, Myanmar, Nepal, and Pakistan. By the end of 2011, the aim is for Skype to be in at least 80% of UNHCR hardship locations and available to more than 3,000 staff members.

All of those are locations where the traditional communication infrastructure may be minimal or in the case of some refugee camps may not even exist.

In reading Skype's blog post and watching their video (embedded below), the technology in the custom version seems to be this:

• Low bandwidth utilization
• Ability to work on many different forms of network infrastructure
• Prioritization of voice over video
• Network-security-friendly, in that:
  • only this version of Skype would be supported on the UNHCR network (and not the publicly downloadable version)
  • UNHCR can restrict the hours in which Skype can be used (to "off-hours" when UNHCR staff can call home)

I can immediately think of other areas where people might want to use a minimal Skype version like this, and indeed the first comment to Skype's post is for someone looking to use it for medical relief efforts in the Congo. Skype's already being used by many crisis organizations... but a low-bandwidth version might only help them use Skype more efficiently. As Skype's Peter Parkes notes in a response, this is an exclusive trial with UNHCR right now... but I do hope Skype will move quickly to make it available to others.

The Enterprise Side?

As I listened to the video, it definitely occurred to me that there are businesses and organizations out there who could also benefit from a low-bandwidth version of Skype. I think, for instance, of shipping companies with limited Internet connectivity to vehicles or ships. Or to companies with distributed offices with very small branch offices with very small Internet connections.

More so, my last bullet about a "network-security-friendly" version... the ability to restrict Skype usage to just this custom version would seem to be of use in certain business settings. Sure, you have some control today with Windows Active Directory settings... but it sounds like this offers more control.

I could see some companies, too, liking the ability to restrict the Skype usage to certain hours... although granted this is just a firewall configuration issue once you have a way to firmly restrict the Skype usage to the one version.

Windows Only?

Of course, I was less thrilled to see that this version is Windows-only. I'll save you my standard rant about Skype's fragmented product strategy since I've ranted about that many times before... and in this case Skype is going a closed trial with a single organization, the UNHCR. If UNHCR uses only Windows systems, then it obviously makes sense for Skype to only develop it for Windows for UNHCR.

However, I know from some limited interaction with people involved with crisis relief and development in remote areas that there is a good bit of work going on with Linux-based solutions. For that reason, it would be good to see this low-bandwidth version available for Linux.

Anyway...

In any event, it's great to see this initiative between Skype and UNHCR. The UNHCR folks do some great work under really difficult situations - and anything that can help aid their communication, even if it is just in helping them stay connected to friends and family - is a wonderful gift. I'm looking forward to hearing more about this initiative in the time ahead.

P.S. And purely FYI to either Tony Bates or whomever wrote the blog post, I think probably more people globally would be familiar with the term "custom" versus "bespoke"... but then again that's the fun of the English language. :-)

---

If you found this post interesting or useful, please consider either:

• subscribing to my email newsletter;
• following me on Twitter; or
• subscribing to the RSS feed

---

Kudos to my friends up at Mitel for posting a customer profile to YouTube, this one of Starwood Hotels & Resorts:

Sure, it's a very professionally-produced video that focuses on marketing messages... and yes, many other companies are already doing this... but having spent six years at Mitel (2001-2007) and having been a strong advocate of the company moving into social media, I just admit that I'm personally pleased to see them doing this.

Looking forward to seeing more...

---

If you found this post interesting or useful, please consider either:

• subscribing to my email newsletter;
• following me on Twitter; or
• subscribing to the RSS feed

---

What is one reason why many people continue using Skype for chat / instant messaging when so many other solutions are out there? Particularly when Skype chat is a closed, proprietary "walled garden" that doesn't interact with IM networks?

After I wrote recently about being a huge user of Skype, Michael Graves asked in the comments why an organization like Voxeo that is so insanely devoted to open standards (and even uses a tagline of "Unlocked Communications") would use something as closed as Skype?

It's a great question, and while I answered his comment, it bears a bit more exploration.

In 3 words, the largest reason for using Skype is this:

persistent group chats

Being a globally distributed company, Voxeo is an IM-centric organization and we set up "group chats" within Skype for pretty much every activity we're doing. Some of those are long-living group chats for communication within various teams or groups of people. Those chats may continue to exist for literally years and have people added and removed to them over time. Some group chats are created for short-term projects or deliverables. And some may be created ad hoc for resolving quick issues - and then disbanded as soon as the issue is dealt with. If a customer has a problem, an alert may be posted in one of our "main chats" and then a "side chat" is formed with the specific group of people who can help right then to resolve whatever the problem will be.

It's a very effective way to work once you get used to it (and learn how to use Skype's ability to notify you of certain types of activity in chats). I have probably 50+ chats open in my Skype client right now, most of which are having little or no traffic at the moment, but a few of which are having active discussions.

The Power of Persistent Group Chats

But what I described as an IM-centric workflow could be accomplished by any chat system... why Skype? This comes down to the difference between typical "group chat" systems and "persistent group chat" systems.

Here's the basic scenario of why this is so powerful:

1. I GO OFFLINE - Perhaps I'm going offline for a meeting. Maybe I'm about to board a plane. Maybe I'm shutting off my system at the end of the day.

2. PEOPLE DISCUSS ITEMS IN MY ABSENCE - The messages in the chat continue to be exchanged, discussions happen, decisions get made, etc., etc.

3. I COME BACK ONLINE - My meeting is over. I landed at my destination. My work day starts. Whatever...

4. I RECEIVE *ALL* THE MESSAGES THAT OCCURRED IN THE CHAT WHILE I WAS OFFLINE - Bingo... I can just scan through everything that happened while I was offline and get caught up on what happened while I was away. Now this sometimes may take a few minutes (for a reason I'll discuss below) and isn't always perfect, but most of the time it works incredibly well.

There is immense collaboration power in this capability. Given that I travel a good bit speaking at conferences I spend a great deal of time on planes. I'll often be working at the airport prior to departure and will be interacting with others via Skype. I'll close my laptop, fly to wherever I'm going, and then open the laptop back up either at the destination airport or at the hotel or office or wherever. Over the course of a minute or two, my Skype client automagically catches up and gives me the full history (subject to a caveat below) of all the discussions that occurred while I was in transit.

Similarly, with globally distributed teams where we may have engineers in Germany, the US and China all collaborating on a project, persistent group chats allow them to rapidly catch up on what occurred when each group was offline.

Of course, if you are offline for a longer period of time, you might come back to literally thousands of messages and want to just "catch up" and mark all old messages as viewed. This was why I was displeased that Skype removed the "Mark All Viewed" button from the Skype 5.0 Beta for Mac client (and I do hope they'll bring it backUPDATE: Skype did bring the feature back in the production release of the Skype 5.0 Mac client).

But why Skype?

But, you say, there are other "persistent group chat" implementations out there... why Skype? Simply because it is the best implementation of persistent group chats we've found so far. Add to that the simplicity of usage, the fact that it has a solid Mac client (and we're a Mac shop), the fact that it can connect from pretty much any location we're in... and the fact that it uses encrypted communication channels.

Having said all this, we're not wed to Skype.... we certainly keep an eye out on other communication tools and have a number of ideas ourselves... if we found something that worked as well and had an open architecture, we'd certainly look at it... but today we use what works - and works well.

The Technology Behind Skype's Persistent Group Chats

If you think about Skype's P2P architecture a bit, the technology behind their implementation of persistent group chats is intriguing. In a typical client/server IM network (like AIM, Yahoo, Jabber, IRC, etc.), the clients are communicating with a server and all the chat messages are stored on the server. Other server-based systems can implement persistent group chats by storing all the messages on the server and then sending them out to clients that re-connect to the server.

But with Skype, there are no servers. Instead, the chat messages get stored in the fabric of the P2P overlay network that interconnects the Skype clients to each other - and more specifically within each of the various Skype clients participating in the group chat.

When your Skype client comes back online, it initiates connections out to other clients that are members of the same chat and requests updates for what messages were sent in the chat while your client was offline. I don't know the exact number of clients your client will reach out to, but conversations with folks from Skype in the past seemed to indicate your client would reach out to a maximum of 15 other clients to find out what was in the chat. (Assuming there are more than 15 people in the chat. If not, obviously it only reaches out to those clients in the chat.)

For EACH group chat that you have.

So if you have a lot of Skype group chats, like I do, you can understand why Skype might trigger security systems at hotels when it goes off to do its initial sync with other Skype clients, purely by the sheer volume of network connections it opens up.

This does bring up one caveat with Skype that I referenced above. Depending upon the size of the chat and the availability of all participants, the full history may not be immediately available. If you are in a chat with 4 people, and the other 3 are offline when you come back online, you won't see the history until others come online. If one other person is online, you will get the history from that other client... which may be the full history, depending upon whether that client was online all the time. You see where I'm going with this... it may take a bit for you to get the full history.

In larger chats, I've seen less of an issue with this because odds are that more people will be online at any time and so your client can receive updates (although there is an edge case that I'll write about sometime). In smaller chats, though, I've seen update issues like this.

All in all it is an intriguing implementation from a technology point-of-view... as someone working with networks for years, I admit to being fascinated by it all. :-)

P.S. It's amusing to also look at what I wrote about Skype's persistent group chats back in January 2007... little did I know how much I would come to use them!

---

If you found this post interesting or useful, please consider either:

• subscribing to my email newsletter;
• following me on Twitter; or
• subscribing to the RSS feed

---

Great stats out of the Silicon Alley Insider's Chart of Day for this past Tuesday:

There was a 31% increase in active smartphones over the Thanksgiving weekend compared to the week prior...

The SAI post explains the chart and how the data was gathered:

Cool info to see!

(Although the paranoid security guy inside of me is admittedly wondering how many of the apps on my iPhone include Flurry's "analytics software" and what exactly it is sending to them... )

---

If you found this post interesting or useful, please consider either:

• subscribing to my email newsletter;
• following me on Twitter; or
• subscribing to the RSS feed

---

When Skype released Group Video Calling with Skype for Mac 5.0 Beta (see my earlier written review and video review), one of the major questions was:

What will happen to group video calling when the free trial ends?

Skype representatives didn't provide a clear answer at the time.

Now we know the answer...

You get another free trial.

And judging from the email sent by Skype, I'm guessing this means "you will keep getting free trials until we figure out precisely what business model works and/or that the technology is all set":

We'll see how long this continues...

---

If you found this post interesting or useful, please consider either:

• subscribing to my email newsletter;
• following me on Twitter; or
• subscribing to the RSS feed

---