Unlimited Plugins, WordPress themes, videos & courses! Unlimited asset downloads! From $16.50/m
by
FREELessons:48Length:7.6 hours

Next lesson playing in 5 seconds

Cancel
  • Overview
  • Transcript

9.3 Downloading Files With FTP

Along with HTTP, another commonly used protocol is FTP (File Transfer Protocol). So in this lesson, we're going to learn how to traverse FTP sites—getting file listings and downloading a remote file. To do so, we'll use the ftplib module.

1.Introduction
2 lessons, 11:32

1.1
Introduction
02:03

1.2
Prerequisites
09:29

2.Python Building Blocks
6 lessons, 1:08:07

2.1
Introduction to the Interpreter
09:51

2.2
Numbers
10:53

2.3
Strings
14:36

2.4
Lists
11:33

2.5
Standard Input and Formatting
12:00

2.6
Building a Tip Calculator
09:14

3.Controlling the Flow
7 lessons, 1:20:10

3.1
Conditional Statements
12:47

3.2
Looping With For
09:36

3.3
The Range Function
10:41

3.4
Looping With While
13:06

3.5
Creating Functions: Part 1
11:55

3.6
Creating Functions: Part 2
08:49

3.7
Building an Average Calculator
13:16

4.Common Data Structures
4 lessons, 46:49

4.1
Lists, Stacks, and Queues, Oh My!
11:45

4.2
Dictionaries
10:10

4.3
Iterating Data Structures
09:41

4.4
Building a Sentence Analyzer
15:13

5.Application Structure
7 lessons, 1:15:12

5.1
Modules
09:08

5.2
Packages
11:24

5.3
Classes
09:53

5.4
Attributes
09:34

5.5
Methods
12:01

5.6
A Special Calculator: Part 1
13:36

5.7
A Special Calculator: Part 2
09:36

6.Collections
7 lessons, 46:55

6.1
What Are Comprehensions?
06:32

6.2
List Comprehensions
06:08

6.3
Dictionary Comprehensions
06:38

6.4
Map
05:45

6.5
Filter
06:31

6.6
Lambdas
05:21

6.7
Generators
10:00

7.File I/O
6 lessons, 48:51

7.1
File Basics
06:50

7.2
Reading Entire Files
07:49

7.3
Navigating a File
08:32

7.4
Writing to Files
07:22

7.5
Reading and Writing to Files
09:15

7.6
Reading and Writing Complex Objects
09:03

8.Networking
5 lessons, 43:48

8.1
Introducing the Socket
04:39

8.2
Getting a Remote IP Address
06:42

8.3
Handling Socket Errors
07:58

8.4
Create a Socket Server
16:04

8.5
Create a Socket Client
08:25

9.Connecting to Network Services
3 lessons, 34:27

9.1
Getting the Current Time With NTP
10:38

9.2
Getting Websites With HTTP
12:57

9.3
Downloading Files With FTP
10:52

10.Conclusion
1 lesson, 02:08

10.1
Goodbye
02:08


9.3 Downloading Files With FTP

When you start to deal with network protocols and you start talking about HTTTP, fairly soon, you're gonna start talking about FTP as well. So while HTTP is nice to serve up HTML and content that you can show to the end user, FTP on the other hand, is the File Transfer Protocol, which is very handy to be able to save and retrieve data from network sources, and then be able to upload files, download files, and traverse folder structures in a very useful way out on the web. So if you need or want to learn more about file transfer protocol in depth you can take a look at all sorts of different sources. I just pulled up one very quickly from Wikipedia. You can read about file transfer protocol here. You could go and take a look at the RFC that once again describes the entire protocol in and of itself and you can write a very elegant full end to end solution in sockets but once again let's skip past all of that stuff and use some libraries or modules that Python has for us. Now for this lesson as an example we're going to take a look at the Linux Kernel Archives now odd's are If you are a Python user, you're probably familiar with Unix or Linux based systems, not to shun our Windows folks out there, I'm a Windows guy as well, but I have found in my experience that a lot of the Python users, are typically Unix and Linux platform users as well. And if that's the case you are probably with the Linux Kernel Archives, which is basically a place where all of the versions of the Linux kernels are stored, so that you can download and retrieve them, learn more about them, and get some updated information about them. Now you can go out to the website and peruse through these things and actually start to download them this way or you can programmatically get access to them using the File Transfer Protocol or FTP version of that site which is ftp.curl.org. So if I just go to ftp.curl.org in my web browser in my case I'm taking about Chrome, I will be able to see some of the structures here. Now your browser may or may not do this very well, but Chrome seems to do a pretty good job. So I can search and navigate through the FTP site for these kernels through my browser. So I could go into pub, and I can see a whole list of folders that are found here. So let's say I wanted to go into Linux. And then I wanted to go into kernel. Now in here you're going to see several different folders, and the interesting folders are the ones out of the bottom that have version numbers on them. These are the versions of the kernels that you can get from the FTP site, ftp.kernel.org. So if I go into the latest one, 4.x you're gonna see a whole slew of files in here, mostly change logs and if you get down to the bottom you're going to see, all of the actual kernels that are zipped up, so that you can download and use them. Now, I'm gonna go through a very simple example that's going to traverse and show you all of the files that are found within this directory using Python. So what I'm gonna do here is I'm actually going to copy this path right here. I'm gonna copy this, and I'm gonna use this in my little application. All right, so here we are back in my text editor, and we're going to start by importing another lib. And yes, I'm gonna be Ftplib. Now once again If you do have this installed, you can just continue to follow along. If you don't have this installed, then you're going to have to install this using pip or some other package manager utility, just like we've done in a previous lessons using pip install ftp lib. So let's go ahead and start by creating a simple function here and all I wanna do is I want to list all the files that are found in that specific directory. So the first thing that I wanna do is I'm gonna name this, so I'm gonna say list_files. And within here, all I want to do is create a new instance of an ftp client. So we'll say ftp_client, Is gonna be equal to ftplib.ftp. And then we need to specify the actual URL that we're trying to get to. So this would be the base URL that we're trying to get to. And in our case, that's ftp.kernel.org. So that's the base URL that we're attempting to get to. Now there's two more parameters in here, those additional parameters are actually a username and password. Now a lot of FTP sites will require username and password access for you to be able to do certain operations. So those could be limiting you to only downloading, only uploading, only perusing or changing directories or things like that. But a lot of FTP sites out there allow anonymous access. The ftp.kernel.org site is one of them that allows anonymous access. So we are simply going to say that we are wanting to do this as anonymous, and then as a password we can just throw pretty much anything in there. Typically we're going to be passing in maybe an email address so I could just put in my email address jensondp@gmail.com and then we are good to go. So now we have a new instance of an FTP client and then the first thing that we wanna do is we want to change our working directory or CWD, to the directory where we have all the files that we want to find. And the way that we do that is by issuing the FTP command cwd, change working directory, and we wanna change our working directory to the location that we found all those files in. So this is where we want to be going. Now once we get there, we want to retrieve a list of all the files that are there. We don't wanna download them, at least not yet. We just want to retrieve a list of files. So to do that, we are going to use the ftp_client command of dir, directory. So it's going to retrieve a list of all of the folders and files that are found within that current directory. Now I'm going to save those off and say files are equal to the result of that directory command and then I'm simply going to print files. And then, when I'm done with that, I'm simply going to close or quit my connection. So I'm gonna say ftp_client.quit, like that. So let's go ahead and save this. Now let's give ourselves a little bit of a mechanism here to run our application like that, and then we can go ahead and execute list_files. Now once again you can make this a little bit more adaptive and you can provide input like an FTP site for the command line, but in this case I wanted to be very specific about where I wanted to go and connect to, the location I wanted to go within that FTP site and then the operation that I wanted to do, so let's go ahead and save this. As the 8-ftp.py, let's come back over to our terminal. And let's go ahead and execute. Python 8-ftp.py. Let's execute and as you could see here, I got a list of all of the files that are found within that location pretty cool. But, another cool thing would be what if I could download one of these programatically? Because typically, when you're dealing with an FTP site, you're really not just traversing, you want to interact with it, you wanna download, you wanna upload, and you wanna be able to interact with those things. So, what if I wanted to take one of these files and I wanted to download them onto my machine? So, what I can do is I'll go back to Chrome here for a second, and lets just find one of these files that I think looks like a good one. So lets pick this change log right here. So lets see what this looks like, okay so we have this basic file right here. So what I would like to do is download this file to my local machine. So this file here, I'm gonna copy it's name, copy. Let's go back over to our little application, so instead of just listing the files, this time, we're gonna list and actually download one of those files. So after we have listed those files, what I wanna do now, is I actually want to download that particular file. So in order to do that I am going to use my ftp_client again and I'm going to issue another command, retrieve binary, so retrbinary, and I'm going to issue an FTP request called RETR. And then I'm going to say what file I wanna retrieve, and I wanna retrieve that ChangeLog right there. But just retrieving that file is not gonna be enough. I need to be able to save it somewhere. So, I'm going to create another variable here and I'm gonna call this changelogfile is gonna be equal to open and I'm simply gonna create a file here and we're going to call this, and I'll actually just call it the same thing here. Copy and we'll call this Changelog.txt and I want to be able to have write access to this once I am actually done downloading and writing it to a file. And then all I have to do is say changelogfile.write. And then once I am done writing that file I am going to want to close that file, so I'll say changelogfile.close. And then I will once again quit my FTP client connection. So let's go ahead and save that. So really with just a couple extra lines here after I have listed all the files there. I want to go ahead and retrieve a file. I wanna write it locally I wanna close that file and then I will close or quite my ftp_client connection. So let's go ahead and make sure that's all saved. Let's come back over here to our terminal. We'll clear this out. Let's go ahead and rerun our application. So now I have completed everything here. I've listed out all the files. And if I list my local files I see a changelog.txt so if I take a look at this file you're gonna see I have all of the content that was found within the file up on web server and if I bring that back up again, you're gonna see all these same data right here. So now, you have a nice little utility that you can use to go and traverse different locations within an FTP site, and as well as being able to download content from it using anonymous access.

Back to the top