Sockets Part I

This post gives an overview of how computer programs can chat with each other. I can't claim to have any great knowledge in this area, I'm just interested enough to try to learn by writing about it. So my first warning is that there are probably mistakes. My second is that this will be a basic overview, my assumption is that anyone reading this knows very little of the area.

First off, there are some obvious and important things to point out.

You need to know the location
If I was writing a program and I wanted two separate classes / functions etc. to interact I would give one a reference to another and let it call what ever it wants. It's pretty simple. With two separate programs it's not that simple because there is no way of knowing where in memory the other program is, and in reality it's probably in the memory of another computer running in some other location in the world. That's the first problem, I need to know where the program is so I can chat with it. This seems to basic to point out but it's good to make things clear.

The solution to this is the IP address and port number. I can use the IP address to identify any computer in the world. Once I know, there are thousands of ports on that computer with each one capable of having a program listening on it. So if I want to chat with another program, I need to identify the machine it is on with the IP address and what port number its listening at.

You need to agree how to transport the data
The messages sent between the programs need a transport mechanism, some low level piece of software that moves it around. For the example below I'm using TCP as the transport layer. TCP guarantees me that a message will be received at the cost of performance. Another option is UDP, which does not guarantee delivery but benefits from being quite quick because of this. The general rule I've come across here is use UDP when you want performance and TCP when you want reliability. That's why VOIP tends to use UDP and web browsers tend to use TCP.

You need to be talking the same language
Chatting within a single program gives you all of the advantages of types. One things gives a number to another, you can be guaranteed that the other will receive a number and can do all sorts of numerical things with it. When chatting between two programs, there are no numbers or strings, only streams of bytes. Each byte represents a character encoding from a character set and both programs need to agree on the character set that they will use.

There's lots to learn in this area (see here http://www.joelonsoftware.com/articles/Unicode.html or here http://www.tbray.org/ongoing/When/200x/2003/04/26/UTF) but the important thing is that both programs are using the same character set.

Working through an example
To keep this simple I want to show as little as possible in each step. First is to show that the two programs can chat. I'll create a C# server that will listen for a connection, greet a client that connects by saying 'Hello' and shuts down. Sounds trivial, but there's quite a bit going on.

static void Main()
{
  var thisMachine = Dns.GetHostAddresses("")[0];
  var server = new Socket(thisMachine.AddressFamily,       SocketType.Stream, ProtocolType.Tcp);
    server.Bind(new IPEndPoint(thisMachine, 4510));
  server.Listen(10);

  var client = server.Accept();
  client.Send(Encoding.ASCII.GetBytes("Hello"));
}

And a python client that connects and displays the received message.

        s = socket.create_connection(('', 4510))
        print s.recv(5)
        s.close()

Let's look at what happens in more detail

            var ipAddr = Dns.GetHostAddresses("")[0];

This will get the localhost addresses ("" tells is to use local host). The addresses returned here will be all of the available addresses for this machine, both IPV4 and IPV6, and I'm just picking the first one off the list.

            var server = new Socket(ipAddr.AddressFamily, SocketType.Stream, ProtocolType.Tcp);

Next step is to create a socket. This bit is important because there are two types of sockets, server sockets and client sockets. The socket we are creating on this line is a server socket and it will be responsible for listening and hooking other programs up with client sockets. This socket is setup to chat over TCP using streams.

            server.Bind(new IPEndPoint(ipAddr, 4510));

Telling the server socket where to listen...

            server.Listen(10);

...and to listen for anyone that wants to chat. The number 10 here is the number of clients that we will allow to back up before going kaput. It's very unlikely that we will reach 10 because the server socket does nothing other than listen and...

            var client = server.Accept();

...accept incoming requests. The call to accept here is blocking, so at this stage the server will wait until another program connects to the end point that it is listening to. This will be done by the python client.

            s = socket.create_connection(('', 4510))

Once our python client connects to the correct location, our server can move on.

                Side Note: It's probably obvious that if you use an incorrect IP address or port number that you won't be able to locate the server. What may not be that obvious is that looking for an IPV4 address when the server is at an IP6 address will mean you're looking at the wrong place. When I get the server local address above I pick the first one off the list so I don't know whether it's an IPV4 or IPV6 address. I get around this by using create_connection() rather than create() from the socket in the python program. create_connection() tells it to look at all available address on the machine.

            client.Send(Encoding.ASCII.GetBytes("Hello"));

The server has created a client to talk to the python client from Accept() and uses that client to send the greeting in bytes, because that's the highest form of communication between two programs. Note that the encoding is ASCII, I'm using ASCII because that's the default in python. So we send the string "Hello", encoded in ASCII, as an array of bytes...

                Side Note: Picking the wrong character set can mess things up. If I switch the line that sends the message to be...

              client.Send(Encoding.Unicode.GetBytes("Hello"));

                ... the resulting string in python would be something like 'H<Null>e<Null>l'. This is because Unicode (this is badly named in C#, it should be UTF-16) uses two bytes per character and ASCII only uses one byte per character.

            print s.recv(5)

...and our python client receives 5 bytes and prints the result. And that's it, two programs have chatted.