LWM-Linux/08 - Advanced Linux Administration/System Log Management (SystemD).md

# System Log Management *JournalCTL)

## 1. Introduction to journalctl

journalctl is a command-line utility for querying and displaying logs from the systemd journal. The systemd journal is a centralized logging system that collects and stores logging data from various sources, including the kernel, system services, and applications.

## 2. Basic Usage

### To view all logs:
`journalctl`

### To follow new log entries in real-time:
`journalctl -f`

## 3. Filtering Logs

### By time:
`journalctl --since "2024-01-01 00:00:00"`

`journalctl --until "2024-01-31 23:59:59"`

`journalctl --since "1 hour ago"`


### By service unit:
`journalctl -u nginx.service`

`journalctl -u ssh.service`


### By priority level:
`journalctl -p err`
Priority levels: emerg, alert, crit, err, warning, notice, info, debug

### By kernel messages:
`journalctl -k`

## 4. Output Formatting

### JSON output:
`journalctl -o json`

### Short output format:
`journalctl -o short`

### Verbose output:
`journalctl -o verbose`

## 5. Boot-specific Logs

### Current boot:
`journalctl -b`

### Previous boot:
`journalctl -b -1`

## 6. User-specific Logs

`journalctl _UID=1000`

## 7. Disk Usage and Log Rotation

### View disk usage:
`journalctl --disk-usage`

### Rotate logs:
`journalctl --rotate`

### Vacuum old logs:
`journalctl --vacuum-time=1week`

`journalctl --vacuum-size=1G`


## 8. Remote Journal Access

To access logs on a remote system:
`journalctl -D /path/to/journal/directory`

## 9. Persistent Journal Storage

### Edit /etc/systemd/journald.conf:
`Storage=persistent`

### Restart journald:
`sudo systemctl restart systemd-journald`

## 10. Forwarding Logs to a Central Server

### Install rsyslog:
`sudo apt install rsyslog`

### Configure /etc/rsyslog.conf for forwarding:
`*.* @@central-log-server:514`

### Restart rsyslog:
`sudo systemctl restart rsyslog`

## 11. Security Considerations

- Restrict access to journal files
- Use encryption for remote logging
- Regularly audit and review logs
- Implement log retention policies

## 12. Performance Tuning

Adjust RateLimitInterval and RateLimitBurst in /etc/systemd/journald.conf to balance between logging thoroughness and system performance.

## 13. Integration with Other Tools

journalctl can be combined with other tools like grep, awk, and sed for advanced log analysis:

```
journalctl | grep "error" | awk '{print $1, $2, $3}'
```

## 14. Scripting and Automation

You can use journalctl in shell scripts for automated log analysis and reporting.