👨🏻‍💻 Viktor's notes

1 · Around The Shell

Oct 05, 202412 min read

Introduction

What’s the shell?

Multiple interfaces exist for interacting with computers, such as Graphical User Interfaces (GUIs), voice interfaces, and Augmented Reality/Virtual Reality (AR/VR) interfaces.

While graphical interfaces are user-friendly, they can be limited in terms of control and automation. For full control over your computer, you can use a textual interface called the shell.

The shell allows you to type commands that the computer executes, giving you powerful capabilities to manage files, run programs, and automate tasks.

Different operating systems offer a variety of shells. One of the most common is bash (Bourne Again SHell), which is widely used on Unix-like systems.

Prompt

To start using the shell, you need a program called a terminal or terminal emulator. The terminal provides a window where you can interact with the shell.

When you open a terminal, you’ll see a shell prompt, which is where you type your commands. It often shows information like your username and current directory.

Here’s an example of a shell prompt:

# missing:$ ··································· bash

This prompt indicates that the shell is ready to accept your commands.

Using the shell

Typing a command at the prompt and pressing Enter will execute that command.

You can also pass arguments to commands to modify their behavior or provide input data.

For example, typing the date command displays the current date and time:

# missing:$ ··································· bash
date

After running this command, the shell might output:

Mon Oct  5 14:23:45 UTC 2024

More Shell Commands

Commands are parsed by the shell using whitespace (spaces and tabs) to separate the command and its arguments.

For example:

# missing:$ ··································· bash
echo hello

The echo command prints the arguments you provide. In this case, it will display:

hello

Shell paths

The shell uses the $PATH environment variable to locate executable programs.

$PATH is a list of directories that the shell searches when you enter a command.

# missing:$ ··································· bash
echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

Understanding how to navigate the file system is crucial.

  • pwd: Displays the current working directory.

    pwd

  • cd: Changes the current directory.

    cd /path/to/directory

  • cd -: Switches to the previous directory.

  • ls: Lists the contents of a directory.

    ls

Using these commands, you can move around the file system and see what files and folders are available.

Shell Permissions

File permissions determine who can read, write, or execute a file.

You can view permissions using:

# missing:$ ··································· bash
ls -l ~

The -l option tells ls to use the long listing format, which includes permissions, ownership, size, and modification date.

Common file management commands include:

  • mv: Moves or renames files.

    mv source_file destination_file

  • cp: Copies files.

    cp source_file destination_file

  • mkdir: Creates new directories.

    mkdir new_directory

To learn more about any command, use the man command to access the manual pages. For example:

man ls

Connecting programs

Programs can be connected using input and output streams.

  • Redirection: You can redirect the output of a command to a file or use a file as input.

    • Output Redirection: > writes output to a file (overwrites if the file exists).

      echo "Hello, World!" > hello.txt

    • Input Redirection: < takes input from a file.

      sort < unsorted_list.txt

  • Pipes: The | symbol passes the output of one command as input to another.

    ls -l | grep "^d"

    This command lists all items and then filters to show only directories.

Such connections allow you to chain commands together to perform complex tasks, and you can set up automated scripts.

More about Programs and Pipes

You can combine commands to perform complex tasks.

For example:

  • curl: Transfers data from or to a server.

    curl http://example.com

  • grep: Searches for patterns in text.

    grep "search_term" file.txt

  • cut: Removes sections from each line of files.

    cut -d',' -f1 file.csv

By combining these commands with pipes, you can manipulate and process data efficiently.

A versatile and powerful tool

The shell gives you powerful control over your system.

NOTE

To execute some commands, you may need administrative permissions (superuser).

Root User is the superuser account with full permissions. Be cautious when using it. For executing commands as the superuser, you can use the sudo command (do as superuser).

This command would allow you to update the package list on Debian:

sudo apt update

Also, with superuser permissions, you are able to access kernel parameters in sysfs, like the CPU temperature:

cat /sys/class/thermal/thermal_zone0/temp

Or you can change the brightness of your Linux-based Intel laptop screen:

echo 1000 | sudo tee /sys/class/backlight/intel_backlight/brightness

Always exercise caution when performing administrative tasks to avoid unintended consequences.

Job Control

Interrupting a Job

Sometimes, you might run a command that takes longer than expected, like searching through large directories with find.

To stop a running command, you can use Ctrl-C, which sends an interrupt signal to the process.

But how does this work?

Killing a process

The shell communicates with processes using signals, which are notifications sent to processes to trigger predefined behaviors.

When you press Ctrl-C, it sends a SIGINT (Signal Interrupt) to the process, which usually tells it to terminate.

A Python script that captures SIGINT and ignores it.

#!/usr/bin/env python
# btw, a previous line is a Shebang line - https://en.wikipedia.org/wiki/Shebang_(Unix)
import signal, time
 
def handler(signum, time):
    print("\nI got a SIGINT, but I am not stopping")
 
signal.signal(signal.SIGINT, handler)
i = 0
while True:
    time.sleep(.1)
    print("\r{}".format(i), end="")
    i += 1

This script increments a counter and prints it continuously. It defines a handler for SIGINT that prevents the script from stopping when Ctrl-C is pressed.

When you run this script and press Ctrl-C, you’ll see:

$ python sigint.py
24^C
I got a SIGINT, but I am not stopping
26^C
I got a SIGINT, but I am not stopping

Since the script ignores SIGINT, you can use Ctrl-\ to send a SIGQUIT signal, which forces the script to quit.

30^\[1]    39913 quit       python sigint.py

Additional Signals

  • SIGINT: Interrupt signal (sent by Ctrl-C).
  • SIGQUIT: Quit signal (sent by Ctrl-\).
  • SIGTERM: Termination signal for graceful shutdown.

To send a SIGTERM to a process:

kill -TERM PID  # where PID is the process ID

Signals allow you to manage processes beyond simple starting and stopping.

Pausing and Backgrounding Processes

  • SIGSTOP: Stops (pauses) a process.
  • SIGTSTP: Terminal stop signal (sent by Ctrl-Z).

When you press Ctrl-Z, the shell stops the running process and returns control to you.

You can then:

  • Foreground (fg): Resume the process in the foreground.
  • Background (bg): Continue the process in the background.

List all jobs with:

jobs

Example Session

$ sleep 1000
^Z
[1]  + 18653 suspended  sleep 1000
$ nohup sleep 2000 &
...
$ jobs
...
$ kill -STOP %1
...
$ kill -SIGHUP %1
...

In this example:

  • sleep 1000 runs a command that sleeps for 1000 seconds.
  • Ctrl-Z stops the process.
  • nohup runs a command immune to hangups.
  • jobs lists current jobs.
  • kill -STOP %1 stops job number 1.
  • kill -SIGHUP %1 sends a hangup signal to job number 1.

Signals to Remember

  • SIGKILL (kill -9): Forcefully terminates a process and cannot be caught or ignored.
kill -KILL <PID>
  • For a list of signals, you can use:
kill -l

Understanding signals helps you manage processes effectively.

Terminal Multiplexers

Introduction

Terminal multiplexers allow you to manage multiple terminal sessions within a single window.

Benefits include:

  • Running multiple sessions simultaneously.
  • Splitting the terminal into multiple panes.
  • Detaching and reattaching sessions, keeping processes running even if you disconnect.

Popular terminal multiplexers:

  • tmux
  • screen

Using tmux

tmux is a powerful tool for managing terminal sessions.

Key concepts:

  • Sessions: Independent workspaces that can be detached and reattached.
  • Windows: Like tabs within a session.
  • Panes: Splits within a window, allowing multiple views.

Default keybindings use a prefix (usually Ctrl-b), followed by a command.

Examples:

  • Ctrl-b c: Create a new window.
  • Ctrl-b ": Split the current pane horizontally.
  • Ctrl-b %: Split the current pane vertically.
  • Ctrl-b d: Detach from the current session.

tmux enhances productivity by keeping your environment organized and persistent.

Aliases

Aliases let you create shortcuts for commands, saving time and reducing errors.

The basic format is:

alias alias_name="command_to_alias arguments"

Alias Examples

Common aliases include:

alias ll="ls -lh"           # Detailed list with human-readable sizes.
alias gs="git status"       # Shortcut for 'git status'.
alias v="vim"               # Shortens 'vim' to 'v'.
alias sl="ls"               # Corrects common typo.
alias mv="mv -i"            # Prompts before overwriting files.

These can be tailored to your workflow to increase efficiency.

Aliases Persistence

Aliases are temporary unless saved in a startup file.

To make aliases permanent, add them to:

  • Bash: ~/.bashrc or ~/.bash_aliases
  • Zsh: ~/.zshrc

For example, add to ~/.bashrc:

alias ll="ls -lh"
alias gs="git status"

Then reload the configuration:

source ~/.bashrc

Dotfiles

Dotfiles are configuration files for your shell and other programs, usually starting with a dot (.).

Examples include:

  • .bashrc: Configuration for Bash.
  • .vimrc: Settings for Vim.
  • .gitconfig: Configuration for Git.
  • .ssh/config: SSH client settings.

Customizing dotfiles allows you to personalize your environment.

Organizing Dotfiles

To manage your dotfiles:

  1. Create a dedicated directory (e.g., ~/dotfiles).

  2. Use version control (like Git) to track changes.

    cd ~/dotfiles
git init

  3. Symlink dotfiles to their appropriate locations.

    ln -s ~/dotfiles/.bashrc ~/.bashrc

Benefits of Organizing Dotfiles

  • Easy setup on new machines: Clone your repository and create symlinks.

  • Consistency: Maintain the same environment across multiple systems.

  • Backup and versioning: Keep a history of changes and restore previous configurations if needed.

Organizing dotfiles streamlines your workflow and setup process.

Portability

A common pain with dotfiles is the configurations might not work across machines. Especially if they have different operating systems or shells.

Sometimes configurations should only apply to a specific machine. There are tricks to make this easier:

  1. Use if-statements for machine-specific customizations.
  2. Use includes for machine-specific settings.
  3. Share configurations across programs.
if [[ "$(uname)" == "Linux" ]]; then {do_something}; fi
if [[ "$SHELL" == "zsh" ]]; then {do_something}; fi
if [[ "$(hostname)" == "myServer" ]]; then {do_something}; fi

In a ~/.gitconfig:

[include]
    path = ~/.gitconfig_local

And ~/.gitconfig_local can have machine-specific settings.

For sharing configurations, such as aliases:

if [ -f ~/.aliases ]; then
    source ~/.aliases
fi

Further Reading

To deepen your understanding:

  • Shell Startup Scripts: Learn how shells initialize and read configuration files.
  • Community Dotfiles: Check out other developers’ dotfiles on platforms like GitHub for inspiration.

TIP

Here are my own personal dotfiles that I use on macOS laptop.

Customizing your environment can significantly enhance productivity.

Remote Machines

Accessing remote servers is common in development and operations.

To connect to a remote server using SSH:

ssh username@remote_server

You can also execute a command directly:

ssh username@remote_server command

Example:

ssh foobar@server ls  # This runs `ls` on the remote server.

SSH Keys

SSH keys provide secure, password-less authentication.

  • Private Key: Stored securely on your local machine (e.g., ~/.ssh/id_ed25519).
  • Public Key: Shared with the remote server, added to ~/.ssh/authorized_keys.

Key Generation

Generate a key pair using (a command from the official GitHub documentation):

ssh-keygen -o -a 100 -t ed25519 -f ~/.ssh/id_ed25519
  • -o: Use new OpenSSH private key format.
  • -a 100: Use 100 rounds of key derivation.
  • -t ed25519: Specify the key type.
  • -f: Specify the output file.

Set a passphrase for added security.

Key-Based Authentication

Copy your public key to the remote server:

  1. Using ssh-copy-id:

    ssh-copy-id -i ~/.ssh/id_ed25519 username@remote_server

  2. Manually copying:

    cat ~/.ssh/id_ed25519.pub | ssh username@remote_server 'mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys'

Now you can log in without a password.

Copying Files Over SSH

Transfer files securely using SSH:

  • Simple Copy:

    cat localfile | ssh username@remote_server 'cat > remotefile'

  • Using scp:

    scp localfile username@remote_server:/path/to/destination
scp -r localdir username@remote_server:/path/to/destination

  • Using rsync:

    rsync -avz localdir username@remote_server:/path/to/destination

rsync is efficient for syncing directories.

Port Forwarding

SSH can forward ports between your local machine and a remote server.

Types of Port Forwarding

  1. Local Port Forwarding: Access remote services locally.

    ssh -L local_port:destination_host:destination_port username@remote_server

  2. Remote Port Forwarding: Expose local services to the remote server.

    ssh -R remote_port:destination_host:destination_port username@remote_server

SSH Configuration

Instead of lengthy commands or aliases, use SSH configuration files.

Edit ~/.ssh/config:

Host my_server
    User foobar
    HostName remote_server
    Port 2222
    IdentityFile ~/.ssh/id_ed25519
    LocalForward 9999 localhost:8888

Now connect with:

ssh my_server

Miscellaneous

Enhance your remote working experience:

  • Mosh: A mobile shell that supports intermittent connectivity.
  • sshfs: Mount remote directories locally over SSH.

Shells & Frameworks & More

While Bash is standard, other shells like Zsh offer advanced features.

Advantages of Zsh:

  • Enhanced Globbing: More powerful file matching patterns.
  • Spelling Correction: Corrects minor typos.
  • Improved Completion: More intelligent auto-completion.

Frameworks provide themes and plugins to extend shell functionality.

Popular options:

  • Oh My Zsh: A community-driven framework with many plugins and themes.
  • Prezto: A configuration framework for Zsh.

Useful plugins:

These tools can make your command-line experience more efficient and enjoyable.

Terminal Emulators

Choosing a terminal emulator is vital. Consider:

  • Font
  • Color Scheme
  • Shortcuts
  • Tabs/Panes
  • Scrollback

🆕 Warp. I’ve used it since 2021, and it’s a game-changer.

Experiment with different terminal emulators to find one that suits your preferences and enhances your productivity.

Graph View

Backlinks