Freemors Blog

Musings of an East Coast Techie
Posts tagged as federation

Three Types of Internet

2016-01-07 by Freemor

I am writing this blog entry to explain to those that may not know the three models of doing things on the Internet. And also why it is important to understand them, to pay attention to them, and choose services and software that use the most correct model.

The three models are:

  • Centralized
  • Federated
  • P2P - Peer-to-Peer

They all have their strength and weaknesses and more importantly, they all have an impact on your rights and freedoms.

Centralized

Centralized is the most common. This is your Google, Facebook, Pinterest, Twitter, Amazon, E-bay, Bank, Etc.

The centralized model has Big servers run by private interests (the site owners/Company) located in some place of their choosing which you use a browser or mobile App. to connect to. Typically all data is stored on the remote server.

This model is perfect for things like banking or online shopping. Just like in the real world you go to the place of business to shop or bank. It is also very appropriate for information type websites news,stocks,weather,sports scores,etc.

The important thing to remember about this model is that you do not control the server and therefor you do not control the data on the server. For the sites mentioned above, no biggie. For things like Facebook,Twitter,etc that live and die on user generated content (your stuff, your data) it's a huge biggie. Once your data is on their server it is usually considered "their data". The User Agreements of such site almost always stipulate that they can do what ever they want with what you upload.

The centralized model is also the easiest for the government to spy on, sensor, control, and shutdown. Because all the data on the server is owned but Company X all the government has to do is legally compel Company X to hand it over. In this way encryption like HTTPS is null and void. Governments can also just seize and shutdown servers they don't like. Also if Company X gets tired of running the server it and all your data will just go Poof and disappear from the Internet.

Considering all these things it is easy to see that the centralized model is both the least free (as in your rights and freedoms) and the most fragile. A lot of service providers out there could be whipped off the Internet by one good flood or other disaster happening to their main server.

Federated

This model is less known and understood by the average person today but it is actually the most common model used in the early days of the Internet. In this model instead on one server (or server farm/s) owned by one company there are many small servers that all talk to each other (federate) to provide a service. This model is used for E-mail, IRC, Usenet, XMPP, UUCP (yes I know that is ancient and deprecated), and newer system like pump.io and Tor. The strength of this system lies in the fact that no one owns the system.. sure they may own a server or two but no one owns the whole system. If a server goes down you just switch to another one.

This model is much harder to sensor, shutdown or control. Servers can live in different countries with different laws and governments. Typically the software to run these kinds of servers is small and easier to install and maintain. This means that anyone with a bit of work and understanding can set up an server and become part of the network of servers. If a government wanted to shutdown the service they'd have to block access to every single server, or a majority of them, to make the system unusable. Not so easy. Spying wise it is harder too. If the government compelled Google to hand over all E-mails (you can be pretty confident that they have/are) it doesn't get them any mails going from sue@freeserver.org to bob@ilivefree.net.

Users typically use some sort of "client" software to connect to their server of choice and interact with the system as a whole. They don't have to worry about what server their friend is on because all servers in the system talk to one another. So sue@freeserver.org can email bill@gmail.com no problem, no worries. As you can see from that example the part that comes after the @ actually refers to what server someone is on in the system. The same is true for XMPP addresses, SIP (proper Voip) addresses, webfinger addresses (pump.io), etc.

There is still the problem of your data on their server.. but as a federated system passes the data from server to server people running federated servers tend to act more like custodians of the data then owners of it. People tend to run these types of servers to offer a public service. OK, well not the Google's of the world. But places like Riseup or Ostel.

Peer to Peer (P2P)

In this model the client software is also the server. All clients on the system talk to and can connect to all other servers on the system. These systems are highly dynamic (servers coming and going all the time) and tend to be very connection and bandwidth heavy because everyone has to help move everyone else's data around.

In a P2P system no one owns the data it just lives out there bouncing from client to client. This means that for most P2P systems you have to be willing to give resources to the network. You have to let the P2P network use some of your bandwidth and disk space.

As you can imagine this is the least easy to censor or shutdown model, and also, if it is done right the hardest to spy on. Because of this many people see the P2P model as a freedom and privacy Panacea. But the truth is this isn't the best model for all things. I don't want to be trading huge chunks of bandwidth and disk space just to see what the weather is going to be like tomorrow. Also because of the dynamic nature of the network and the problem of where stuff is stored relative to who is online the P2P model isn't really the best for "store and forward" applications like E-mail. Sure there are things like Bitmessage but if Bob isn't around for a day or two after Sue tries to send him a bitmessage her software will have to try sending it again. If they have really bad timing it could take months for Bob to get the message. Where in a federated system Sue would send the data to her server of choice which would send it to Bob's server of choice which would hold on to it till bob came online.

People in remote locations or developing countries may not have the bandwidth or disk space to share. There are people in the area where I live for whom a P2P system could easily eat their monthly data allotment in a day or two.

Even tho a P2P system that used good encryption for transfer and storage would be very hard to spy on these systems are complicated beasties and are prone to other forms of attack, resource depletion, evil clients that do things like say they'll forward that data but then throw it away thus vanishing it from the network, governments running a ton of clients to analyze the traffic flow and figure out who is talking to who or even who is who, etc.

It is also important to note that many P2P systems like Bittorrent and Bitcoin do nothing to hide your IP address, so there is no anonymity. Many people are confused and think that P2P automatically means anonymous.

Which is Best

There really is no one best model. The important thing is to try and pick the services that are using the right model for the right job and be aware of the trade offs

  • more right but more resources (P2P) - Heavy on bandwidth, CPU time, and disk space but no central server, just other people using the software.
  • No rights but fast, easy and light on resource (centralized) - Where people running the service control everything. The rules, your data, who has access and how, etc.
  • a bit of a mix (federated) where people running the many servers take the bandwidth and resource hit.

Things to watch out for are centralized sites that are trying to own and control your data, and a newer trend of big companies trying to push the workload onto users by using P2P technologies. Netflix has eyed this to take some of the load off their servers by making people watching a show also stream that show to other people watching the show.. great for them.. terrible for your bandwidth.

Pay attention to which model a service is using and you have a much better ideal of how it effects your rights, freedoms, data, bandwidth, and disk space.

Why isn't Instant Messaging like E-Mail?

2015-08-04 by Freemor

or

Who stole my freedom and why

Have you ever wondered why you can e-mail anyone with any e-mail account but you can only chat to people on the same service as you? It's a really good question and the answer will surprise you.

The fact is you, I, Anyone CAN chat (IM) to anyone anywhere. The technology exists and is in use all over the place today. The reason most people don't is because of two things.

A) The big companies don't want you to
B) The big companies don't want you to

I know, I know, A and B look a lot the same but really they are two sides of the same coin. Lets start with A.

The Big Companies Don't want you to (A)

The Internet's big boy Google,Yahoo, Facebook, Msn, etc Want to make money off of you and since the first days of Instant Messaging companies realized it was a "killer feature" everyone raced to have the best chat (IM) application. And they made sure that their chat wouldn't chat to the other guys. Why? Because they want to make money off of you... not just you... you, your friends, their friends, your kids, your kids friends, etc. They want to monetize as many users as they can... and they understand that if all your friends are on "Super X chat" then chances are good that you'll cave to presure and use "Super X chat". Partly because all your friends are on it... partly because all those people being on it creates a lot of "buzz" so you hear about it a lot more. Partly because of branding. It is a potent mix. The big companies understand this and intentionally screw you out of your freedom (to chat to anyone) in a bid to pressure your friends and family to also join. Big companies see IM as the crack that will get not just you but all your friends hooked on them. So they never tell you there is another option. In fact they lie and say it cant be done because of X, Y, and/or Z

Having a lot of people using one Instant messenger creates a LOT of pressure for others to use it. Big companies understand this. It even has a name "The Network effect". The more people there are using a particular instant messenger (or any other social thing) the more people will gravitate to it and the harder it is to leave. Think of it like a social black hole. The bigger it gets the more and faster it pulls people in and the harder it is to escape. For the big companies this means lots and lots of customers and lots and lots of money. For you and your friends this means getting screwed out of your freedom to talk to anyone... because now you can only talk to people on "Super X chat" and the only way to talk to a person not on it... is to get them hooked too.

The big companies don't want you too (B)

The free and open protocol I mentioned before (XMPP) is just that, free and open. None of the big companies control it, much like the protocol that runs E-mail, in fact your username on XMPP looks just like an E-mail address and if you are lucky it can be the same as you E-mail address, Mine is.

Just a quick note here for the everyday users. All Instant Messenger systems are comprised of two parts. The protocol. a definition of how software using the system will talk to one another and the sotware. Now it is understandable that most people don't know this distinction because Skype software speaks the Skype protocol and since Microsoft controls the protocol and wont tell or let others figure out how it works ONLY the Skype software speaks it. Thus the average user never sees the distinction between protocol and software with instant messaging. You do with E-mail because the protocol is open and not controlled by any one company so there are hunderds of different E-mail programs that can all talk to one another.

So if companies were to properly implement XMPP they would lose control, you wouldn't have to be on their network, you wouldn't even have to use their software. They couldn't force ads on you. It would be much harder to track and profile you. They'd lose control of you and the leverage they had over your friends. They also would loose the ability to add and remove features willy nilly. They'd have to try to be as standard as possible so that you didn't suddenly find yourself unable to talk to friends or you'd just leave and go get another account. Imagine if gmail suddenly stopped being able to send mail to Yahoo because of some "feature", no one would put up with that. So instead of being the crack that gets you hooked on the big companies they'd see it as a burden "have to provide to compete" type service and they really don't like those.

With all that it is no wonder that so few people have heard of XMPP. If people knew they'd leave the big boys in droves and never come back.

So How do I do this XMPP thing?

Pretty easily actually:

  • You download one of the XMPP clients that you like.
  • You set up an account on ANY proper XMPP server. (That's just a small list to get you started)
  • You share your favouritename@XMPPserver address with your friends.

But here is the hard part...

You refuse to let Big Companies screw you out of you freedom by using a "popular" Instant messenger. As soon as you do that no one on that service will switch. You loose. They loose. Big company wins and continues to screw people out of their freedom.

It is that last bit, the "Network Effect" that is hindering XMPP as much as the big guys not talking about or using it. People that do know and make the move over to XMPP are often faced with loosing contact with friends. The thing to remember is, if enough people switch then the "Network Effect" will be on our side. The more people there are using XMPP the more people will want to use XMPP. And suddenly it wont matter what IM software you like or what server you are on, you'll be able to talk to anyone anywhere. Just like E-mail.