Linux Server
Goals
- TODO
Concepts
- Bash
- CentOS
- daemon
- Debian
- Extra Packages for Enterprise Linux (EPEL)
- Filesystem Hierarchy Standard (FHS)
- group
- home directory
- init
- journal
- kernel
- KiTTY
- Linux distribution
- login shell
- OpenJDK
- Personal Package Archive (PPA)
- process
- process ID (PID)
- PuTTY
- Red Hat
- Red Hat Enterprise Linux (RHEL)
root
- Secure Shell (SSH)
- services
- shell
- signal
- superuser
systemctl
systemd
- targets
- Telnet
- timer unit
- Ubuntu
- unit
- unit file
- Unix
- user
Language
Library
Dependencies
Preview
Preparation
If you are using Windows, download and install either the PuTTY or the KiTTY SSH client.
Lesson
Linux is an open-source operating system and is one of the most popular server platforms. Since then then it has forked into a multitude of variations, each known as a distribution or distro
. Linux even underlies Google's Android operating system on mobile devices.
Of the most popular Linux distributions there are two main divisions, based upon the package management system they use (explained later in this lesson). Many distributions have variations targeted for server or for personal desktop use.
- Red Hat
- A commercial Linux distribution by the company Red Hat, primarily through their Red Hat Enterprise Linux (RHEL) product. CentOS is a free, community based version of RHEL and is the most popular Linux distribution for web. CentOS is the distribution we will be using in this lesson. See The most popular Linux for Web servers is ....
- Debian
- One of the earliest Linux distributions. The company Canonical Ltd. produces another very popular Debian-based distribution named Ubuntu.
Logging In
Effecting a login to a Linux machine requires a username and a password. Upon login a user is placed in the user's home directory, which by by default is /home/username
. The tilde ~
character is a shortcut way to represent the current user's home directory. The home directory is also stored in the $HOME
environment variable.
SSH
Remote login is usually performed over a secure connection using the Secure Shell (SSH) protocol, which uses port 22 by default. SSH communication is based upon the public cryptography key shared during the first connection with a server. Future connections use this key not only to secure the connection but also to verify the identity of the server, which is why many SSH clients will ask if the server's public key should be stored for later use. The initial transfer of the public key is performed before the server has been identified, so in a very high security situation the public key should itself be transferred in a secure manner before the first connection.
Bash
Communicating with Linux is traditionally performed using a command-line interpreter called a shell. Although you can configure Linux to use your shell of choice, the most popular shell is Bash, which comes as the default on many Linux installations.
Built-In Commands
Bash as with other shells comes with several built-in commands which do not require the installation of a separate program. Here are a few of the most useful ones:
cd
- Changes the current directory. Using
cd
by itself withouth indicating a directory name will change to the user's home directory. echo
- Prints something to the screen; e.g.
echo Hello, World!
exit
- Exits the script or the shell. If you have switched to another user during a session,
exit
will return to the session of the previous user. export
- Sets an environment variable for the current session; e.g.
export FOO=bar
For the variable to be available the next time you log in, you will need to add it to a login script, explained below. help
- Prints help information for a built-in command.
kill
- Sends a signal to end a running process. TODO see below
pushd
/popd
- Changes to a new directory while placing the old one on an internal stack; and returns to the last pushed directory and removes it from the stack, respectively.
pwd
- Prints the working directory.
source
- Executes a script. The full stop or
dot
.
character is a synonym forsource
.
TODO add command-line arguments to all commands above
Scripts
When you first log in, Bash executes several scripts in order:
/etc/profile
/etc/profile
calls all scripts in/etc/profile.d/
~/.bash_profile
~/.bash_profile
calls~/.bashrc
~/.bashrc
calls/etc/bashrc
Thus if you want to add an environment variable for a single user, add the appropriate export
command to ~/.bashrc
. If you want to add an environment variable for all users, the best practices is to create an appropriately named file inside /etc/profile.d/
(e.g. /etc/profile.d/maven.sh
for Maven environment variables such as M2_HOME
). If you add a new script file you must make it executable (e.g. chmod +x /etc/profile.d/maveh.sh
), explained in Permissions below.
TODO tip about using source to reload profile
TODO discuss creating a script to be called by a user, specifying the shell
Environment Variables
Bash allows environment variables to be set using export
, as shown above. TODO talk about placing in .bashrc Here are some of the most common environment variables:
_
- The most recently executed command.
HOME
- The home directory of the logged-in user.
HOSTNAME
- The hostname of the machine.
PATH
- The paths for searching for commands, separated by the colon
:
character. SHELL
- The path to the shell.
USER
- The username of the logged-in user.
To retrieve the value of an environment variable, prefix it with the dollar $
sign, such as $HOME
as you saw above.
Essential Commands
To get around in the shell and navigate the file system you need to be familiar with the following commands, along with some important command-line arguments:
cat [<option>]... [<file>]...
- Prints the contents of files to the standard output. So named because it allows multiple files to be concatenated.
ls [<option>]... [<file>]...
- Lists the contents of a directory.
--all
(-a
)- Lists all directories, even those starting with the full stop
.
character. TODO see below about hidden files -l
- Use a long listing format.
mkdir [<option>]... [<directory>]...
- Creates directories.
more
- Filters output to display information a screenful at a time.
TODO include rm, and note rm -rf
TODO include ln, add note about not putting link in directory form
Users and Groups
Linux controls access to resources more tightly than has traditionally been the case on desktop Windows systems. This is especially desirable in a server environment, where multiple people may enter the system but only certain of them should have access to administrative tasks.
As with most other platforms, the people who access the system are referred to as users. Users may be placed in one or more groups. Linux permissions may be specified for an individual user or for an entire group.
root
Linux, following the tradition of Unix, has a special user named root
which is a superuser—it has permissions to modify anything in the system. The use of the root
user is necessary for initial setup of the system, but best practices recommend that separate users be set up immediately for continued access and system maintenance.
sudo
The program sudo
allows individual users to perform tasks that would normally only be accessible to root
. Simply by prefixing a command with sudo
, a user will be able to perform a task using root
privileges. The user will need to provide his/her password (not the root
password) to gain these privileges. The sudo
command by default performs the command as root
, but using the -u
flag and specifying a username will perform the command as any specified user.
CentOS | Ubuntu | |
---|---|---|
Add User |
| sudo adduser username |
Change Password | sudo passwd username | sudo passwd username |
Give sudo Access | sudo gpasswd -a username wheel | sudo gpasswd -a username sudo |
See Also | How To Create a Sudo User on CentOS | How To Create a Sudo User on Ubuntu |
TODO: note about changing user settings after creation with usermod; see http://linuxg.net/how-to-change-a-user-login-shell-uid-or-homedir-with-usermod/
Files
All files in Linux are placed at some level under a single root /
directory. The Filesystem Hierarchy Standard (FHS) provides a standard set of directories for Linux distributions to follow, although different distributions are at various levels of compliance of different levels of the standard. Here are some of the most important Linux top-level directories; most of them have specialized subdirectories for different types of executable files and data. TODO note about files beginning with a dot
/
- Root directory of the entire file system hierarchy.
/etc/
- System configuration files.
/home/
- User home directories.
/lib/
- Libraries.
/mnt/
- Mounted file such as hard drives and DVD drives..
/opt/
- Optional software packages.
/root/
- Home directory for the
root
user. /usr/
- Contains shareable, read-only data. TODO note /usr/local/bin/ as a location for installing add-on executables
/var/
- Place for
variable
files which are expected to change.
TODO note difference between /usr/local and /opt; see http://unix.stackexchange.com/a/11552
ln -l
for directory foo
and file bar
.Permissions | Links | Owner | Group | Size | Modified Timestamp | Filename |
---|---|---|---|---|---|---|
drwxrwxr-x | 9 | jdoe | jdoe | 4096 | Oct 29 15:38 | foo |
-rw-rw-r-- | 1 | jdoe | jdoe | 1526374 | Oct 2 20:15 | bar.txt |
Permissions
Every file (including directories) has three classes of permissions: for file owner user (u
), for the file group (g
), and for other users (o
). For each class there are three permissions: read (r
), write (w
), and execute (x
). A long directory listing will show file permissions in the first column, as in the following figure:
foo
and file bar
.
drwxrwxr-x 9 jdoe jdoe 4096 Oct 29 15:38 foo
-rw-rw-r-- 1 jdoe jdoe 1526374 Oct 2 20:15 bar.txt
File | Type | User (u ) | Group (g ) | Others (o ) | ||||||
---|---|---|---|---|---|---|---|---|---|---|
Read | Write | Execute | Read | Write | Execute | Read | Write | Execute | ||
foo | d | r | w | x | r | w | x | r | - | x |
bar.txt | - | r | w | - | r | w | - | r | - | - |
There are several commands available to change file permissions. Each of these commands allow the --recursive
(-R
) flag indicating that permissions should be changed recursively for all files in all subdirectories.
chown [<option>]... [<owner>][:[<group>]] <file>...
- Changes the owner of a file; e.g.
chown jsmith bar.txt
setsjsmith
as the owner ofbar.txt
. chgrp [<option>]... <group> <file>
- Changes the group of a file; e.g.
chgrp students bar.txt
sets the group ofbar.txt
tostudents
. chmod [<option>]... <mode>[,<mode>]... <file>...
- Changes the file
mode
bits, that is, the permissions. For the mode, specify the symbol indicating the class: user (u
), group (g
), others (o
), and/or all (a
); followed a symbol to add permissions (+
), to remove permissions (-), or to set permissions (=
); followed by the permissions read (r), write (w
), and/or execute (x
). For examplechmod g+rw foo --recursive
adds read and write permissions for each file's group in the entirefoo
directory hierarchy. If no class is indicated, the default is all (a
).
Processes
As you learned when studying Java threads, each application runs in an operating system process. In Linux each one is given a process ID (PID), which you can use for indicating a process in commands.
Listing Processes
Here are common commands you can use for listing processes.
top [<option>]...
- Shows a dynamically updated list of all running processes. For a more user-friendly dynamically-updating display of processes, you can use the
htop
command, but usually it must be installed separately. See Packages below. On CentOShtop
is available from the EPEL, also explained below. ps [<option>]...
- Produces a static list of processes in various formats. The
ps
switches with the hyphen-
character are different than those without. For exampleu
shows detailed information, while-u
restricts the processes to those of a user. pgrep [<option>]...
- Returns the PID for a process.
Killing a Process
Stopping a process is done by the built-in kill
command, already noted in a section above. The command kill -TERM foobar
(or simply the kill foobar
) sends a signal to the process to terminate. If the process refuses to shut down, you can use -KILL
option which bypasses the application, routing signal to to operating system.
Commands
TODO wget, curl, more
TODO tar, especially with -J for xz
TODO pwd
TODO ln
TODO ln -al; pushd/popd; source (.)
TODO nano, screen
systemd
In addition to the processes you may run manually and that eventually end, for Linux to function there must be many processes that stay running, or run periodically. For example, if it were not for the SSH daemon sshd
running, there would be nothing listening when you try to connect to a Linux server using an SSH client. For these sort of services, the major Linux distributions have migrated to a system of programs and daemons called systemd
.
Like Maven, systemd
is declarative. The services to run are defined as targets, and individual services declare dependencies. When it comes time to run a service, systemd
can examine the dependencies to determine in which order services must be started. The individual jobs that systemd
controls are called units.
systemctl
The primary program to control systemd
is called systemctl
. It has several commands for managing units:
list-units
- Lists all configured units. This is the default command; you can simply enter
systemctl
to see the available units. status <unit>
- Gives the status of some unit.
start <unit>
- Starts a unit.
stop <unit>
- Starts a unit.
restart <unit>
- Stops and then restarts a unit.
enable <unit>
- Configures a unit to start automatically after reboot.
disable <unit>
- Stop a unit from starting automatically after reboot.
reload <unit>
- Reloads a unit's configuration without stopping the unit.
daemon-reload
- Reloads the configurations for all
systemd
units.
Unit Configuration
Adding a new unit (or configuring an existing one) is accomplished by adding a unit file to the directory /etc/systemd/system
. The base name of the unit file indicates the name of the unit, while its extension indicates the type of unit. For example to configure a service for the Tomcat server, you would create a file named /etc/systemd/system/tomcat.service
. (See the section on Tomcat for more details and a sample service unit file.)
Defining, starting, and enabling a service thus includes the following steps:
- Create the unit file
/etc/systemd/system/example.service
. - Reload the systemd configurations using
systemctl daemon-reload
. - Start the service using
systemctl start example.service
. - Enable the service for automatic starting with
systemctl enable example.service
. - User
systemctl status example.service
to ensure the service is started and enabled.
Packages
TODO prose
TODO mention groups
CentOS | Ubuntu | |
---|---|---|
Update Package Lists | yum check-update | sudo apt update |
Upgrade Installed Packages | sudo yum update | sudo apt upgrade |
Find a Package | yum search string | apt search string |
View Package Information | yum info package | apt show package |
Install Package | sudo yum install package | sudo apt install package |
Install Package Group | sudo yum groupinstall "group" | sudo apt-get install group |
Remove Installed Package | sudo yum remove package | sudo apt remove package |
Add Repository | sudo yum install repository | sudo add-apt-repository ppa:repository/ppa |
See Also | Package Management Basics: apt, yum, dnf, pkg |
TODO note about how to list users
TODO list some common packages to install, e.g. nano (not available by default on CentOS 7)
Applications
Linux applications are installed using the package system for the distribution. Here are some popular applications, along with special installation and configuration details.
Git
The installed version of Git on both CentOS and Ubuntu is out of date. You'll need to either install it manually or pull it from an alternate repository. Don't forget that each user must still set his/her Git username and email.
CentOS
wget https://www.kernel.org/pub/software/scm/git/git-2.10.1.tar.xz
tar -xf git-2.10.1.tar.xz
cd git-2.10.1
make configure
./configure --prefix=/usr #Necessary for CentOS.
sudo make install
See How To Install Git on CentOS 7.
Ubuntu
You can install the latest version of Git directly from the PPA ppa:git-core/ppa
.
sudo add-apt-repository ppa:git-core/ppa
sudo apt-get install git
Java
CentOS
Although it not too difficult install Oracle's version of Java on Linux, the installation packages require a procedure outside the standard package-management command, including a manual download, cookie support, and acceptance of a license agreement. More straightforward is installing the OpenJDK, an open-source implementation of Java. OpenJDK comes in two variations: java-1.8.0-openjdk
which only includes the JRE, and java-1.8.0-openjdk-devel
which includes the entire JDK.
sudo yum install java-1.8.0-openjdk-devel
See How To Install Java on CentOS and Fedora.
Ubuntu
TODO; see https://www.digitalocean.com/community/tutorials/how-to-install-java-with-apt-get-on-ubuntu-16-04
TODO note alternate PPA for getting Java 8 on 14.04, e.g. WSL; see http://askubuntu.com/q/464755
TODO maybe simplify by creating a table with package names
Maven
The installed version of Maven on both CentOS and Ubuntu is out of date. It is straightforward to install it manually, however, and gives a good example of installing standalone applications that do not need to be built (as Maven uses Java).
Install Maven Files
Download and install Maven in a separate /opt/apache-maven-x.x.x/
directory, and create a symbolic link from /opt/maven/
to this directory. This allows us to easily upgrade Maven by pointing /opt/tomcat/
to a different directory in the future.
wget http://mirrors.ibiblio.org/apache/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
sudo tar -xf apache-maven-3.3.9-bin.tar.gz --directory /opt/ #sudo needed for writing to /opt
sudo ln -s /opt/apache-maven-3.3.9/ /opt/maven #create symbolic link for home directory
sudo ln -s /opt/maven/bin/mvn /usr/local/bin/mvn #add system path link to Maven executable
Define Maven Environment Variables
Use nano
or another editor to create the /etc/profile.d/maven.sh
file below, which will set the Maven environment variables for the entire system:
Tomcat
The version of Tomcat available from the repositories is several versions behind, so you'll want to install Tomcat manually to get the latest version. This will also give you experience with creating users, setting permissions, and configuring a systemd
unit.
TODO prose about /opt/tomcat
Create Tomcat User and Group
Create a system user tomcat
with group tomcat
, with no login privileges and a home directory in /opt/tomcat/
. The actual directory will be created later. See Creating Group and User for Tomcat RHEL/CentOS for a discussion of installation variations.
sudo useradd -r -s /sbin/nologin -d /opt/tomcat/ tomcat
Install Tomcat Files
Download and install Tomcat in a separate /opt/apache-tomcat-x.x.x/
directory, and create a symbolic link from /opt/tomcat/
to this directory. This allows us to easily upgrade Tomcat by pointing /opt/tomcat/
to a different directory in the future.
wget http://download.nextag.com/apache/tomcat/tomcat-8/v8.5.6/bin/apache-tomcat-8.5.6.tar.gz
sudo tar -xf apache-tomcat-8.5.6.tar.gz --directory /opt/ #sudo needed for writing to /opt
sudo ln -s /opt/apache-tomcat-8.5.6/ /opt/tomcat #create symbolic link for home directory
Configure Tomcat Permissions
Give the tomcat
group access to the entire Tomcat installation, but only give the tomcat
user write access to certain files in case there is a security breach. See Apache Tomcat 8 Security Considerations for more details on Tomcat security.
sudo chgrp --recursive tomcat /opt/tomcat/ #give tomcat group access to Tomcat installation
sudo chmod g+r --recursive /opt/tomcat/ #give tomcat group read access to Tomcat installation
sudo chmod g+rwx /opt/tomcat/conf/ #allow tomcat group to add files to the configuration directory
sudo chown --recursive tomcat /opt/tomcat/logs/ /opt/tomcat/temp/ \
/opt/tomcat/webapps/ /opt/tomcat/work/ #make tomcat user owner of working directories
Set Up Tomcat systemd
Unit
Use nano
or another editor to create the /etc/systemd/system/tomcat.service
file below:
Define Tomcat Manager User
If you plan on deploying via the Tomcat Manager App or as part of a Maven build, define a Tomcat user in /opt/tomcat/tomcat-users.xml
having the manager-gui
and/or manager-script
roles, respectively. (See Migration Guide - Tomcat 7.0.x: Manager Application for a list of default supported roles.) This allows the identified user to deploy applications via the web UI, or via automated build scripts (such as within Maven), respectively. Set an appropriate manager username and password.
TODO mention how to change port number
Start Tomcat Service
Now that you've defined a service for Tomcat, tell systemd
to reload the service files and then start Tomcat. You may also want to enable the Tomcat service so that it starts automatically when the system boots.
sudo systemctl daemon-reload #refresh systemd
sudo systemctl start tomcat #start Tomcat
sudo systemctl enable tomcat #configure the Tomcat service to start on boot (optional)
Test Tomcat
If you were successful, you can access the default Tomcat splash page in a web browser by accessing port 8080
, e.g. http://localhost:8080/
.
Deploying to Tomcat
Autodeploy a WAR via webapps/
The easiest way to deploy a web application to Tomcat is simply to copy the WAR file to the Tomcat webapps/ directory. Tomcat will automatically open the WAR file (which by default will be readable by Tomcat even if owned by root
) and install the web application in its own context, using the WAR base filename as the context root. For example adding a foo.war
file would use /foo/
as the context path for any servlets in the application; if a servlet in the web application used the annotation @WebServlet("/bar")
, on the servlet would be accessible at /foo/bar
.
cd foo #change to the foo project directory
sudo cp target/foo-1.0.0-SNAPSHOT.war /opt/tomcat/webapps/foo.war
Deploying via the Manager App
TODO warning about unsecure connection
TODO
TODO note about being able to use the web management interface to upload a WAR
TODO see http://tomcat.apache.org/maven-plugin-2.2/
TODO: add tomcat7-maven-plugin; see scratch application; see http://tomcat.apache.org/maven-plugin-2.0/tomcat7-maven-plugin/
TODO describe setting password in external configuration file; stress importance of credentials being outside the POM
TODO describe mvn tomcat7:deploy/redeploy
Review
Summary
Gotchas
- TODO mention gotcha about globs being expanded before sudo kicks in; see https://stackoverflow.com/q/40289476/421049
- Don't confuse the
ps
switches with a hyphen-
character and those without. - Do not place custom
systemd
unit files in/usr/lib/systemd/system
; put them in/etc/systemd/system
instead. - Do not enable remote access on Tomcat without enabling a secure connection over SSL/TLS, or your password could be compromised.
- If your WAR filename contains a version number or other information, don't forget to rename the WAR file if autodeploying to the Tomcat
webapps/
directory.
In the Real World
Think About It
Self Evaluation
- The most popular Linux distributions fall into which two categories?
- What is the default port of SSH?
- At what point in SSH communication would it be theoretically possible to compromise security?
- What would be the default home directory for the user
jdoe
? - What command will print the current user's home directory?
Task
A dedicated virtual CentOS server has been created for you, and you will be emailed the IP address and root
credentials. Configure the server and deploy your Booker servlet on the server running on Tomcat.
- Log in as
root
. - Immediately change the
root
password. TODO important: use a strong password. - Create a separate user with
sudo
capabilities. TODO important: Log out and only log on as the other user under normal circumstances. - Update the system with the latest packages.
- Install Git, Maven, and Tomcat.
- Deploy your Booker servlet on Tomcat
- Clone your
booker-server
repository somewhere in your individual use home directory. - Package your
booker-server
project using Maven. - Auto deploy the WAR file by copying it to the Tomcat
webapps/
directory.
- Clone your
See Also
- How to choose the right Linux distro (InfoWorld)
- How To Read and Set Environmental and Shell Variables on a Linux VPS (DigitalOcean)
- How To Create a Sudo User on CentOS (DigitalOcean)
- How To Create a Sudo User on Ubuntu (DigitalOcean)
- An Introduction to Linux Permissions (DigitalOcean)
- How To Use ps, kill, and nice to Manage Processes in Linux (DigitalOcean)
- 10 basic examples of Linux ps command (BinaryTides)
- Understanding and Using Systemd (linux.com)
- Understanding Systemd Units and Unit Files (DigitalOcean)
- How To Use Systemctl to Manage Systemd Services and Units (DigitalOcean)
- Package Management Basics: apt, yum, dnf, pkg (DigitalOcean)
- How To Install Apache Tomcat 8 on CentOS 7 (DigitalOcean)
References
Resources
Acknowledgments
- Some symbols are from Font Awesome by Dave Gandy.