# Useful *NIX Shell Commands for Web Developers

So, why *NIX and why do you need to go to the console?

According to the statistics of W3Techs, Unix is used by 68% of all the websites whose operating system they know. This means that if you are web developer, your code is most probably running on a Linux server. And at least you need to know how to configure and debug your code on Unix and Linux systems. Let's find out what you need to know to feel comfortable in the command line.

## The Basics

The basic *NIX command consists of three components:

• command or program to run
• options to alter or specify the behavior of the command
• arguments or input data that is needed to run the command

For example, if you need to get a list of files in the directory /var/www, you need to run the command ls with the argument /var/www. To add the size of files to the output, you need to add the -s option, and the final command will look like this:

 1 ls -s /var/www 

### I/O Redirections and Pipelines

Many *NIX commands use text input and output which you can operate with, and the great feature of this is that you can send the output results of the command to a file using redirect, or even pass the output of one command to the input of another command using the pipelines. For example, we can output the command from the previous example to a file:

 1 ls -s /var/www > /var/www/files.txt 

This command will create or erase file /var/www/files.txt and output a list of files in the /var/www directory. Here is a list of standard I/O redirections and pipelines:

• > Redirect output from a command to a file on disk. The file will be erased and overwritten.
• >> The same redirect, but appending the output file.
• < Get input to command from a file.
• | Pass the output of one command to the input of another command.
• tee Both redirect output to a file and pass it to the next command in the pipeline.

### The Main Commands

To get manual pages for a command, run man. Manual pages follow a common layout and may include name, synopsis, description, and usage examples. This will display the documentation for the chmod command:

 1 man chmod 

To execute some commands like saving configurations or rebooting processes, you need to run them as the super user. To do this, you need to prepend sudo to your command:

 1 user@server:/var/www$chmod 777 log  2 chmod: changing permissions of ‘log’: Operation not permitted  3 user@server:/var/www$ sudo chmod 777 log  4 sudo chmod 777 log 

If you need to execute a bunch of commands as a super user, you can use su, or switch user command.

### Searching Files

There is the find command to search for files in a directory hierarchy. This command is very powerful and can search for files and directories by name, access permissions, date, and size.

Find all directories with the “logs” name in the /var/www directory using the -type option:

 1 find /var/www -type d -name logs 

To search PHP files in the current directory, add the -name option:

 1 find . -type f -name "*.php" 

Find files with defined permissions using the -perm option:

 1 find . -type f -perm 0777 -print 

Find all files which are greater than 500MB:

 1 find / -size +500M 

Of course, you can combine all of those options in one command, and this is only the basics of the find command, which is a very powerful tool for searching files. Use manual pages to get more information.

### Manipulating Files and Folders

There are five main commands for manipulating files and folders in *NIX system:

• touch is used to change timestamps on existing files and directories, but also this is the easiest way to create new file
• mkdir to make directories
• cp to copy files and directories
• mv to move or rename files and directories
• rm to delete files and folders

The next example will create a file index.html, copy this file to the new directory in /var/www, and remove the source file.

 1 root@localserver:~# touch index.html  2 root@localserver:~# mkdir /var/www/newdir  3 root@localserver:~# cp index.html /var/www/newdir/  4 root@localserver:~# rm index.html 

Another great command is ln, which is designed to make links between files. The command ln is often used to create a symbolic link for enabling a virtual host:

 1 sudo ln -s /etc/apache2/sites-available/newvirtualhost.com.conf /etc/apache/sites-enabled/ newvirtualhost.com.conf 

### Change Access Permissions

To change the file owner and group, use chown. Don't forget to grant ownership to the apache user when you are creating a new virtual host of your web application:

 1 sudo chown -R www-data:www-data /var/www/newvirtualhost.com 

Sometimes cache or log directories of your application must be writable for all users so you need to change access modes to 777 with the chmod command. Add the -R option to add permission to all nested files and folders.

 1 sudo chmod -R 777 /var/www/private/cache 

If you just want to make a file executable, use chmod with the +x option.

 1 sudo chmod +x /var/www/private/backup.sh 

To view files in the console, you can use the cat command. With cat, you can concatenate files' contents using extra parameters, and you can also use mask in filenames.

 1 cat /etc/apache2/apache2.conf  2 cat /etc/apache2/apache2.conf /etc/apache2/ports.conf  3 cat /etc/apache2/mods-enabled/* 

But the cat command will get you confused very fast, because it shows output in raw format without any paging—so it is inconvenient to use with log output. To get a filter for paging through text one screenful at a time, you should use the more or less commands, which are much of a muchness.

 1 less /etc/apache2/apache2.conf  2 cat /etc/apache2/mods-enabled/* | less 

Another useful command is tail, which is created to output the last part of files. This command is perfect to look through log histories. By default, this tail command prints the last 10 lines, and you can change this number using the -n parameter.

 1 tail /var/log/apache2/error.log  2 tail -n 25 /var/log/apache2/error.log 

But if you have, for example, a bunch of log files, you need something more powerful to make a proper search. Something like grep—a program that reads from standard input, tests each line against a pattern, and writes to standard output the lines that match this pattern. By using it in combination with cat and pipelines, you will get what you want.

If you want to filter text-lines of output, you can use the grep command:

 1 grep notice /var/log/apache2/error.log  2 cat /var/log/apache2/*.log | grep "shutting down" 

As you can see, grep is good for using in pipelines. In this example, the last command will output all lines containing the “shutting down” string from log-files.

### File Editing

If you want to edit text files in console mode, you can use one of the three most popular text editors:

• GNU nano, a small and friendly default text editor, which is a perfect choice for basic tasks
• Vim, an improved programmers' text editor, which is most powerful, but complex for beginners
• mcedit, a full-featured windowed editor from Midnight Commander, which is easy to use but not installed by default on *NIX systems

Compare them and make your choice:

 1 nano /var/log/apache2/error.log  2 vim /var/log/apache2/error.log  3 mcedit /var/log/apache2/error.log 

### Archiving

Sometimes you need to back up or compress some data on your server.

The most common archiving utilities are tar and zip. Notice that the zip command may not be installed on your server by default.

You can create an archive with the following commands:

 1 tar -zcvf archive-name.tar.gz directory-or-file-name  2 tar -jcvf archive-name.tbz2 directory-or-file-name  3 zip archive-name.zip directory-or-file-name 

If you want just to see a list of files in the archive, you can use the -l option for both tar and unzip:

 1 tar -ztvf archive-name.tar.gz  2 tar -jtvf archive-name.tbz2  3 unzip -l archive-name.zip 

Or extract some source files:

 1 tar -zxvf archive-name.tar.gz  2 tar -jxvf archive-name.tbz2  3 zip archive-name.zip 

If you want to schedule your scripts to run periodically, you need to use the Cron utility, which is driven by a cron table—a configuration file that specifies the shell commands to run periodically on a given schedule. And the command to maintain cron tables is crontab.

Calling crontab with option -l will show your cron table.

Also, the -u option is provided to specify the name of the user whose crontab is being used. If you are going to run tasks of your web application, it is better to edit crontab for user www-data.

 1 user@server:~$sudo crontab -lu www-data  2 # m h dom mon dow command  3 */5 * * * * php5 /var/www/yii do/tasks >> /var/www/tasks.log  4 00 15 * * 1-5 /var/www/backuper.sh  In this output, you can take a look at an example of a cron table. As you can see, every line is scheduled by minute, hour, day of month, month, and day of week. Every field may be an asterisk, which means every value of the field. Also you can use sets and ranges using commas and hyphens. Following a range with a slash specifies skips of the number's value through the range. In this example, the first command will run every five minutes, and the second command will run from Monday to Friday at 15:00. To edit this list, run crontab with the -e key instead of -l. The cron list will be opened in your default editor. Use the -r option to clear the cron list. ### Performance Monitoring Command top shows system summary information and provides a dynamic real-time view of running system processes. Press Shift-M to sort processes by memory usage, or Shift-P to sort by CPU usage.  1 top - 21:33:02 up 308 days, 21:24, 1 user, load average: 0.00, 0.01, 0.05  2 Tasks: 87 total, 1 running, 86 sleeping, 0 stopped, 0 zombie  3 %Cpu(s): 0.3 us, 0.0 sy, 0.0 ni, 99.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st  4 KiB Mem: 501800 total, 471348 used, 30452 free, 49672 buffers  5 KiB Swap: 4194300 total, 56192 used, 4138108 free. 149488 cached Mem  6   7  PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND  8 16269 www-data 20 0 348592 38884 12044 S 0.0 7.7 0:02.42 php5  9 26533 www-data 20 0 409516 38488 24312 S 0.0 7.7 1:00.04 php5-fpm  10  1076 mysql 20 0 887824 32748 1616 S 0.0 6.5 276:46.59 mysqld  11  862 syslog 20 0 256612 31428 368 S 0.0 6.3 32:45.88 rsyslogd  12 18901 root 20 0 105632 4316 3248 S 0.0 0.9 0:00.04 sshd  13 25393 www-data 20 0 87356 4312 1564 S 0.0 0.9 4:46.92 nginx  14 27846 memcache 20 0 328464 3828 252 S 0.0 0.8 1:04.30 memcached  To display the amount of free and used memory in the system, use the free command. Add the -h option to display output fields in human-readable format.  1 user@server:~$ free -h  2  total used free shared buffers cached  3 Mem: 490M 453M 36M 23M 46M 140M  4 -/+ buffers/cache: 265M 224M  5 Swap: 4.0G 54M 3.9G 

Another useful command is df, which is a command to report file system disk space usage. You can call it with the -a option to show all the file systems of your server. Also, don't forget to add the -h option for human-readable format.

 1 user@server:~$df -ah  2 Filesystem Size Used Avail Use% Mounted on  3 /dev/vda1 20G 6.3G 13G 34% /  4 udev 235M 4.0K 235M 1% /dev  5 tmpfs 50M 344K 49M 1% /run  ### Command Line History You can use the !! command to repeat the previous command, or use sudo !! if you forgot to run a command with sudo.  1 user@server:/var/www$ chmod 777 log  2 chmod: changing permissions of ‘log’: Operation not permitted  3 user@server:/var/www\$ sudo !!  4 sudo chmod 777 log 

If you forgot the syntax of commands or are feeling lazy about typing a large command query, you can use history to display your command history. It is good to combine this command with strings filter commands like grep, tail and others to find exactly what you want.

 1 history | tail  2 history | grep crontab  3 history | egrep -i 'ssh|ftp' 

## Conclusion

Using the console is not rocket science. Unix and Linux systems are easy to understand and use because of their simple design and good documentation. I hope this article will make you pretty comfortable with the command line and bring you to the next level of managing your web applications with the command line.

If you have any questions or you want to share your favourite console commands, don't hesitate to leave a comment below the article.