8.4 Create a Socket Server
It's finally time to start to use sockets for communicating over the network. In this lesson, we are going to build a basic server that will wait for a connection from a client and echo any incoming messages back. This basic exercise is the foundation for creating a socket-based server application.
1.Introduction2 lessons, 11:32
2.Python Building Blocks6 lessons, 1:08:07
3.Controlling the Flow7 lessons, 1:20:10
4.Common Data Structures4 lessons, 46:49
5.Application Structure7 lessons, 1:15:12
6.Collections7 lessons, 46:55
7.File I/O6 lessons, 48:51
8.Networking5 lessons, 43:48
9.Connecting to Network Services3 lessons, 34:27
10.Conclusion1 lesson, 02:08
8.4 Create a Socket Server
Now it's time for us to dig a little bit deeper into the world of sockets. Now, to this point, we've done some fairly nice little parlor tricks. Well, they're not really parlor tricks, but when it comes to sockets, being able to retrieve some local host information, some IP address and system name, that's great. Being able to figure out the IP address of some remote system, that's great too, but it's really very, very high level when it comes to sockets. So sockets really start to shine when we begin doing things like creating client server applications. Where we can manage connections and receive data and send data and do all those types of things. And you might say well, why is that important? Well, you are surrounded in your day to day activities by client server applications that at some point, down at some low level of networking uses some sort of socket. And I mean that by saying every time you send somebody a text message or every time you connect to Facebook or every time you go to LinkedIn or to TuttsPLus or you send somebody a chat on instant message, all of these things are at some form or another a client server based application. That is using some sort of socket communication for the most part down at the lower levels. Now we're not going to write any sort of big complicated applications like that, but this is a very important thing to understand and be able to do, should you need to In your day to day activities. So, what we want to do is we want to emulate something like that, where we want to be able to create a low level socket connection, from a server perspective, and accept connections and data from clients, and then process that data and do something with it. Maybe it's sending us commands, maybe it's a chat application. It doesn't really matter, but the basic premise is going to be the same. We want to create a server accept connections, get data and do something with it. So let's start by creating a very simple server. So the first thing that we're gonna do is we're gonna import socket, just like we normally would so that we normally have, at this point. Now the first thing that we need to do is we need to create a new instance of a socket. The cool thing about this as we go through this lesson, you're gonna find very quickly that the process of creating and using sockets is very speakable. So, I can talk to it very easily using just regular terminology and you're gonna find that it's very simple to follow along and pretty easy to remember once you start getting used to doing this. So I'm gonna create a new socket and I'm gonna call it S and I'm gonna say socket. Socket is my function I want to use to create a new instance of a socket and this is gonna take in two parameters. The first one being what address family do we want to be working with for this particular socket and address family meaning are we talking about IPv4 addresses, or IPv6. Now there's other options in here, but those are the two most common ones. Now we could use either one, I'm gonna stick with IPv4, simply because that's probably the easiest [LAUGH] one for all of us that we're most familiar with to use, but you could support version six, should you so choose. Now the way that we specify that is actually quite simple, we can use a constant that is within these socket class and that's actually going to be af_inet, now af_inet corresponds to IPV4. Now if you wanna use IPV6 at this point you would throw a six on the end of it but like I said for this particular course we're gonna stick with v4. Now the second parameter we're gonna pass in is the socket type or connection type and that means basically connection less or connection based. Now most client server applications are typically connection based. There's examples out there of connection less. The most common ones that you're probably used to behind the scenes of your favorite applications are gonna be connections based. So we're gonna use another constant at this point, and this is gonna be SOCK_STREAM. So this is gonna be a streamed based connection meaning we're gonna open up a connection, we're gonna stream data back and forth. This is gonna be a bidirectional stream, so I can send and receive data on the same connection, which is kinda nice. I don't have to open up multiple connections. So now that I have this new socket, I need to bind it to an address, which means I need to specify what address and what port I wanna be communicating on locally. Now if you have some sort of public facing server, you could use a public facing IP address and port. And that's probably what you would wanna do. But since I'm working locally on a laptop I only have access to my private network and things like that, and I'm really only working on one machine here. The trick that I'm going to use here is I want to bind to my loop adapter. I want to be using 127.0.0.1 just to make things easy. So what I'm gonna do here is I'm gonna say socket.bind is gonna be the function. It's gonna take a single parameter but it's in two parts. And it's going to be the IP address, so I'm gonna say 127.0.0.1 as a string. And then I also need to specify the port as an integer. Now the port is basically any port you can use on your machine. I would avoid any of the more commonly used ports like 80, or 443, or things like that, unless you are creating an application specifically to use those ports, because those are generally accepted to be used for other things like web applications, or websites, or things like that. So, you could pick just about anything you wanted here. Let's say maybe 8,000. But you could pick pretty much anything you want. All right, so now we're bound to an IP address and a port. We now want to start listening for external clients to make connections to us. And the way that we do that, believe it or not, is using the listen function. See what I'm talking about? The nomenclature, the terminology that we're using here, maps one to one basically with the functions on the socket to make this fairly easy to do it. Now listen does take a parameter. It basically takes a queue depth. And what is a queue depth? Well, a single machine or even a server farm, always has an unlimited amount of resources. You have limited CPU power, you have limited memory, limited disk space. Nothing is unlimited. You can make an argument for the cloud where things are expandable, but for the most part we're talking about limited resources here. So I cannot really open this up to an infinite number of client connections and be able to handle all that, because I'm gonna quickly run out of processing power. My machine's gonna crawl down to a halt and then ultimately probably shut down, which is obviously not what we want to do. So the reason we specify a queue depth here is to say, if I specify a depth of five, that means I'm gonna be able to queue up five requests at a time so I can process one, or if I make this multi-threaded which I'm gonna save for another course. We could do, we could process a couple things at a time, but there is gonna be things queued up in this listen function. So if I have a queue depth of five, and six requests come in, or four connections to come in, the first five are going to get queued up, and that sixth one is going to fall off as basically like a connection denied type of a thing. So they're gonna have to retry, using some sort of retry logic along the way. And then once one of those requests is done and being processed and released. Then my queue is gonna go down to four and I will accept another connection in to that queue. So that's the basic process in a first in, first out type of order. But since we're writing a very simple client server application here I'm simply going to say one, I really only want to handle one connection at a time because that's really all I am going to be doing. So now my application is going to sit here and it's going to start listening for connections from clients so, once a connection comes in I want to accept that connection and I wanna do something with it. So I'm going to, believe it or not, use the accept function, so I'm going to accept one of those connections. And as a result of that I'm gonna get two things out of this. I am gonna get a connection and I'm going to get an address. And that connection is going to allow me to communicate by directionally from the server to the client, both send and receive. And the address is gonna tell me who's connected to me. Two very useful pieces of information. So typically, at this point, I like to print out something to say where I'm at or who's connected to me. So I'm gonna say connected to client, and then I'm simply going to put address on the end of here so I can see what is the client address that is working with right now? So once I've received that connection, I need to start doing something. I need to start receiving data, processing it and then maybe sending data back or something like that if we're talking about a bidirectional application. So how do we do that? Well one of the complicated things to do with low level socket programming is to really know how much data is being sent to you, because you really don't, you never know that. You never really know up front how much data is being sent to you. Now obviously if you're control of both the client and the server, and you specify what sort of commands and payloads you're gonna be dealing with you'll know roughly the size of the data coming across but it's really impossible to ever know exactly. So the way that we typically do this is we create a loop that's going to run. I'm gonna say infinitely but I'm gonna show you why it's really not infinitely, in just a moment. And then we are going to take that data that's coming in in chunks, and so we no longer receive any more data. So we know once we stop receiving the data, that request is done I can process it and I can move on. So the way that we do that is we're going to create a simple loop here, a while loop and so we're gonna put some sort of truthy statement in here. It can be pretty much anything as long as it resolves to true so I'm just gonna basically say one so that's going to make it so that I'm running this loop infinitely until I forcibly break out of here. And then I wanna start receiving data from the client. So I'm gonna say data = connection.recv. So recv is the receive function. And this is gonna take in a buffer size. So how large are the buffer chunks of data that I wanna be receiving from my client? In this case, since we're gonna create a very simple application, I'm only gonna basically be sending strings back and forth. But it could be commands or similar things like that. I'm gonna keep it relatively small in bytes. And I'm gonna say 1024 should be way more than I'm going to need. But obviously, you can make this larger and smaller to fit your needs. Now, do remember that it is important to size this properly because if you make this too small, then this connection is gonna have to work really hard to receive a lot of chunks. But if you also make it too large, then you're gonna be receiving large chunks of data that you're gonna have to process, before you can do anything with it. So just kind of keep that in mind when you're sizing this. I usually start with 1024 and then we can adjust the size after that. Now the first thing that I wanna do once I receive data is I wanna see did I actually receive any data? Because if I didn't then I just want to kick out of this loop and be done. So in this case I'm gonna say if there is not data, then I wanna break, fairly simple. So if I'm done receiving all that data. If the data that I received in my buffer is empty, then I'm just gonna get the heck out of here. Once I've done that, I am going to simply print out what I received from the client. So I'm gonna say received from client. And I'm going to put on here, data. So now we're going to be able to see what I received from the client. And then at this point, you could do some sort of data processing, you could run commands on the server, you could send messages to other clients or whatever have you. But at this point, I'm simply going to echo this back to my client. So I'm going to say connection.send, and I wanna send that data back to the client. So this is pretty simple. I'm gonna receive data. If there's nothing there I'm gonna break out of my loop. If there is data there then I'm going to print received from client and I'm gonna throw my data out there so you can see what I got. And then I'm going to echo that back to the client that I'm currently connected to. Now if I do break out of this while loop, one thing to remember is that you are gonna want to close that connection. Because if you don't close that connection, you're gonna wind up not freeing up the resources associated with that connection, as well as tying up that port so that you won't be able to use that port anymore. And then you're gonna have to go in there and forcefully close whatever process is tied up to that port. So at this point, you're gonna simply say connection.close. Now I'm gonna save that. Now one other thing I just thought about now that I'm going through this is I'm gonna come up after this bind statement. And I'm gonna put a print in here. And I'm gonna say print listening for connections like this and save. So what's gonna happen here when I run this application and you're gonna see If I didn't put this line in here, once I hit this listen line here, it's just gonna sit here and it's gonna look like the application's hung, but it really isn't. So I'm gonna make sure that I print this out so you can see it's actively listening for connections and then we can process anything as we come in. So let's make sure this is all saved. And let's give this a little test. So I'm gonna come over to a client here, or to a terminal, and I'm gonna run Python and I'm going to run my server application, and as you can see here now I'm listening for connections. So as I mentioned before if I didn't put in that log line this would just sit here, and then it would be a little confusing, what's going on. Now I want to test this out, well the way that I"m gonna test this is out is I'm gonna open another terminal, and I'm going to telnet into that connection. Now, If you have a Unix or Linux based machine, or Mac like I have here, you're probably gonna have to tell that client automatically installed. With Windows you may or may not. So if you don't, you can easily go out and find some sort of free, open source telnet client, very simple to use, In this case, I am simply going to use this from the command line. I'm going to say, telnet. I'm going to telnet into my IP address that I am listening on. And then I am listening on port 8000. So, I am going to telnet into 125.00.1 to port 8000. So, let's go ahead and run that, so now you can see, I'm connected to local host from my client, and on my server you see I'm connected to client that's sending me data from 127.0.0.1 on port 60400 And 44. So from a client perspective, telnet here, it's basically picking some sort of random open port that it can communicate from and connecting to port 8000 on the server. So what happens now? Well, I can start typing in things. I can say, hello there. And I'm going to see, received from client: hello there, and I'm gonna get it echoed back on my side over here. Now it's hard to tell that this was received from the server and not a duplicate but in the next lesson I'm gonna show you another little trick where we can fix that up. So you can continue to type in commands here, what's your name? So it echoed back, what's your name? I asked you first and you could have a nice little conversation with yourself but it's really gonna go nowhere cuz it's just gonna echo data back. But that is the basic process of creating a server application using Python and low-level sockets. Now one thing that is important to remember as I mentioned before it's always good to clean up your connections and close everything out so you can reuse ports From a server perspective. So, in order to do that, it's quite simple, for my client I can close out this entire connection. So, when I'm in telnet on my Mac here, I can use ctrl ], and it's going to stop the connection here and I go back to my telnet prompt and I can simply say quit. And once I quit, I'm going to close the connection from the client and ultimately also close it from the server perspective. So there you go, that's the basic process of creating a socket based server using Python.