missing_class

Command-line environment

menu_book

Table of Contents

Job control

Pausing and backgrounding processes

Sample session

$ sleep 1000
^Z
[1]  + 18653 suspended  sleep 1000

$ nohup sleep 2000 &
[2] 18745
appending output to nohup.out

$ jobs
[1]  + suspended  sleep 1000
[2]  - running    nohup sleep 2000

$ bg %1
[1]  - 18653 continued  sleep 1000

$ jobs
[1]  - running    sleep 1000
[2]  + running    nohup sleep 2000

$ kill -STOP %1
[1]  + 18653 suspended (signal)  sleep 1000

$ jobs
[1]  + suspended (signal)  sleep 1000
[2]  - running    nohup sleep 2000

$ kill -SIGHUP %1
[1]  + 18653 hangup     sleep 1000

$ jobs
[2]  + running    nohup sleep 2000

$ kill -SIGHUP %2

$ jobs
[2]  + running    nohup sleep 2000

$ kill %2
[2]  + 18745 terminated  nohup sleep 2000

$ jobs

Terminal Multiplexers

tmux has the following hierarchy of sessions, windows, and objects

Sessions

Windows

Panes

References

Aliases

alias alias_name="command_to_alias arg1 arg"

Aliases

# Save a lot of typing for common commands
alias gs="git status"
alias gc="git commit"
alias v="vim"

# Save you from mistyping
alias sl=ls

# Overwrite existing commands for better defaults
alias mv="mv -i"           # -i prompts before overwrite
alias mkdir="mkdir -p"     # -p make parent dirs as needed
alias df="df -h"           # -h prints human readable format

# Alias can be composed
alias la="ls -A"
alias lla="la -l"

# To ignore an alias run it prepended with \
\ls
# Or disable an alias altogether with unalias
unalias la

# To get an alias definition just call it with alias
alias ll
# Will print ll='ls -lh'

Dotfiles

Some other examples of tools that can be configured through dotfiles are:

Common edit is to add a program to the path environment variable:

export PATH="$PATH:/path/to/program/bin"

Best way to manage your dotfiles is to store them in their own folder, symlink to them, and version control the folder. Benefits:

Use if statements to vary config for machine specific customisations

if [[ "$(uname)" == "Linux" ]]; then {do_something}; fi

# Check before using shell-specific features
if [[ "$SHELL" == "zsh" ]]; then {do_something}; fi

# You can also make it machine-specific
if [[ "$(hostname)" == "myServer" ]]; then {do_something}; fi

If the configuration file supports it, make use of includes to store machine specific settings which could be tracked in separate repositories. For example, a ~/.gitconfig can have a setting:

[include]
    path = ~/.gitconfig_local

if you want different programs to share some configurations. For instance, if you want both bash and zsh to share the same set of aliases you can write them under .aliases and have the following block in both:

# Test if ~/.aliases exists and source it
if [ -f ~/.aliases ]; then
    source ~/.aliases
fi

Remote machines

Secure shell (ssh)

To login as foo at bar.mit.edu server:

ssh foo@bar.mit.edu

Key generation

Key based authentication

ssh will look into .ssh/authorized_keys to determine which clients it should let in. To copy a public key over you can use:

cat .ssh/id_ed25519.pub | ssh foobar@remote 'cat >> ~/.ssh/authorized_keys'

A simpler solution can be achieved with ssh-copy-id where available:

ssh-copy-id -i .ssh/id_ed25519.pub foobar@remote

Copying files over SSH

Port Forwarding

In many scenarios you will run into software that listens to specific ports in the machine. When this happens in your local machine you can type localhost:PORT or 127.0.0.1:PORT, but what do you do with a remote server that does not have its ports directly available through the network/internet?.

This is called port forwarding and it comes in two flavors: Local Port Forwarding and Remote Port Forwarding (see the pictures for more details, credit of the pictures from this StackOverflow post).

Local Port Forwarding Local Port Forwarding

Remote Port Forwarding Remote Port Forwarding

The most common scenario is local port forwarding, where a service in the remote machine listens in a port and you want to link a port in your local machine to forward to the remote port. For example, if we execute jupyter notebook in the remote server that listens to the port 8888. Thus, to forward that to the local port 9999, we would do ssh -L 9999:localhost:8888 foobar@remote_server and then navigate to locahost:9999 in our local machine.

SSH Configuration

We have covered many many arguments that we can pass. A tempting alternative is to create shell aliases that look like

alias my_server="ssh -i ~/.id_ed25519 --port 2222 - L 9999:localhost:8888 foobar@remote_server

However, there is a better alternative using ~/.ssh/config.

Host vm
    User foobar
    HostName 172.16.174.141
    Port 2222
    IdentityFile ~/.ssh/id_ed25519
    RemoteForward 9999 localhost:8888

# Configs can also take wildcards
Host *.mit.edu
    User foobaz

An additional advantage of using the ~/.ssh/config file over aliases is that other programs like scp, rsync, mosh, &c are able to read it as well and convert the settings into the corresponding flags.

Note that the ~/.ssh/config file can be considered a dotfile, and in general it is fine for it to be included with the rest of your dotfiles. However, if you make it public, think about the information that you are potentially providing strangers on the internet: addresses of your servers, users, open ports, &c. This may facilitate some types of attacks so be thoughtful about sharing your SSH configuration.

Server side configuration is usually specified in /etc/ssh/sshd_config. Here you can make changes like disabling password authentication, changing ssh ports, enabling X11 forwarding, &c. You can specify config settings in a per user basis.

Miscellaneous

Shells & Frameworks

Alternatives to bash: e.g. the zsh shell is a superset of bash and provides many convenient features out of the box such as:

Frameworks can improve your shell as well. Some popular general frameworks are prezto or oh-my-zsh, and smaller ones that focus on specific features such as zsh-syntax-highlighting or zsh-history-substring-search. Shells like fish include many of these user-friendly features by default. Some of these features include:

NB frameworks may slow down your shell, especially if the code they run is not properly optimized or it is too much code. If an issue: profile it and disable the features that you do not use often or value over speed.

Terminal Emulators

Along with customizing your shell, it is worth spending some time figuring out your choice of terminal emulator and its settings. There are many terminal emulators out there (here is a comparison).

Since you might be spending hundreds to thousands of hours in your terminal it pays off to look into its settings. Some of the aspects that you may want to modify in your terminal include:

TeX and pandoc

Exercises

Job control

  1. From what we have seen, we can use some ps aux | grep commands to get our jobs’ pids and then kill them, but there are better ways to do it. Start a sleep 10000 job in a terminal, background it with Ctrl-Z and continue its execution with bg. Now use pgrep to find its pid and pkill to kill it without ever typing the pid itself. (Hint: use the -af flags).

  2. Say you don’t want to start a process until another completes, how you would go about it? In this exercise our limiting process will always be sleep 60 &. One way to achieve this is to use the wait command. Try launching the sleep command and having an ls wait until the background process finishes.

    However, this strategy will fail if we start in a different bash session, since wait only works for child processes. One feature we did not discuss in the notes is that the kill command’s exit status will be zero on success and nonzero otherwise. kill -0 does not send a signal but will give a nonzero exit status if the process does not exist. Write a bash function called pidwait that takes a pid and waits until said process completes. You should use sleep to avoid wasting CPU unnecessarily.

Terminal multiplexer

  1. Follow this tmux tutorial and then learn how to do some basic customizations following these steps.

Aliases

  1. Create an alias dc that resolves to cd for when you type it wrongly.

  2. Run history | awk '{$1="";print substr($0,2)}' | sort | uniq -c | sort -n | tail -n 10 to get your top 10 most used commands and consider writing shorter aliases for them. Note: this works for Bash; if you’re using ZSH, use history 1 instead of just history.

Dotfiles

Let’s get you up to speed with dotfiles.

  1. Create a folder for your dotfiles and set up version control.
  2. Add a configuration for at least one program, e.g. your shell, with some customization (to start off, it can be something as simple as customizing your shell prompt by setting $PS1).
  3. Set up a method to install your dotfiles quickly (and without manual effort) on a new machine. This can be as simple as a shell script that calls ln -s for each file, or you could use a specialized utility.
  4. Test your installation script on a fresh virtual machine.
  5. Migrate all of your current tool configurations to your dotfiles repository.
  6. Publish your dotfiles on GitHub.

Remote Machines

Install a Linux virtual machine (or use an already existing one) for this exercise. If you are not familiar with virtual machines check out this tutorial for installing one.

  1. Go to ~/.ssh/ and check if you have a pair of SSH keys there. If not, generate them with ssh-keygen -o -a 100 -t ed25519. It is recommended that you use a password and use ssh-agent , more info here.
  2. Edit .ssh/config to have an entry as follows
Host vm
    User username_goes_here
    HostName ip_goes_here
    IdentityFile ~/.ssh/id_ed25519
    RemoteForward 9999 localhost:8888
  1. Use ssh-copy-id vm to copy your ssh key to the server.
  2. Start a webserver in your VM by executing python -m http.server 8888. Access the VM webserver by navigating to http://localhost:9999 in your machine.
  3. Edit your SSH server config by doing sudo vim /etc/ssh/sshd_config and disable password authentication by editing the value of PasswordAuthentication. Disable root login by editing the value of PermitRootLogin. Restart the ssh service with sudo service sshd restart. Try sshing in again.
  4. (Challenge) Install mosh in the VM and establish a connection. Then disconnect the network adapter of the server/VM. Can mosh properly recover from it?
  5. (Challenge) Look into what the -N and -f flags do in ssh and figure out what a command to achieve background port forwarding.

Solutions

Job Control

  1. $ sleep 10000
^-Z
[1]  + 12388 suspended  sleep 10000
$ bg %1
[1]  + 12388 continued  sleep 10000
$ pgrep -af sleep
12388 sleep 10000
$ pkill -f sleep
[1]  + 12388 terminated  sleep 10000

  2. $ sleep 30 &
[1] 12606
$ pgrep sleep | wait; ls

    Bash function pidwait:

    pidwait() { 
  wait $1
  return 0 
}
$ sleep 30 &
[1] 12721
$ pgrep sleep | pidwait
[1]  + 12721 done       sleep 30
[2]  - 12736 done       pgrep sleep

##

Edit this page.