pfSense/OPNsense | Elastic Stack v7.9+ | Ubuntu 20.04+
pfELKCore installation.
Requirements
Ubuntu Server v20.04+
pfSense v2.4+ or OPNsense 19+
Primary Installation Method
Scripted Install
A. Download Script
wget https://raw.githubusercontent.com/pfelk/pfelk/main/etc/pfelk/scripts/pfelk-installer.sh
B. Make Script Executable
chmod +x pfelk-installer.shC. Run Script Installer
sudo ./pfelk-installer.sh D. Proceed to Section e3 below:
Alternate Installation Method
Manual Install
(a) - Preparation
a1. Configure Date/Time Zone
sudo timedatectl set-timezone EST
a2. Disable Swap
sudo swapoff -a
(b) - Prerequisites
MaxMind (optional), apt-transport, ELK repositories, ELK, GPG signing key, Java 14 LTS
Add Elastic Stack Repository
b1. Add Elastic Stack (Elasticsearch, Logstash and Kibana) Repository
echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list
b2. Download and install the public GPG signing key (Elastic Stack)
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
Update System Repositories
b3. Update Repositories
sudo apt update
Install apt-transport
b4. Download and install apt-transport-https package
sudo apt install apt-transport-httpsInstall Elastic Stack
b5. Install Elastic Stack (Elasticsearch, Logstash and Kibana
sudo apt install elasticsearch; sudo apt install kibana; sudo apt install logstash
Configure Kibana
c1. Configure Kibana
sudo nano /etc/kibana/kibana.ymlc2. Modify host file (/etc/kibana/kibana.yml)
server.port: 5601
server.host: "0.0.0.0"
Configure Logstash
c3. Create the following directories
sudo mkdir -p /etc/pfelk{conf.d,config,logs,databases,patterns,scripts,templates}c4. Download conf.d files
sudo wget https://raw.githubusercontent.com/pfelk/pfelk/main/etc/pfelk/conf.d/01-inputs.conf -P /etc/pfelk/conf.d/ sudo wget https://raw.githubusercontent.com/pfelk/pfelk/main/etc/pfelk/conf.d/05-apps.conf -P /etc/pfelk/conf.d/ sudo wget https://raw.githubusercontent.com/pfelk/pfelk/main/etc/pfelk/conf.d/20-interfaces.conf -P /etc/pfelk/conf.d/
sudo wget https://raw.githubusercontent.com/pfelk/pfelk/main/etc/pfelk/conf.d/30-geoip.conf -P /etc/pfelk/conf.d/
sudo wget https://raw.githubusercontent.com/pfelk/pfelk/main/etc/pfelk/conf.d/35-rules-desc.conf -P /etc/pfelk/conf.d/
sudo wget https://raw.githubusercontent.com/pfelk/pfelk/main/etc/pfelk/conf.d/36-ports-desc.conf -P /etc/pfelk/conf.d/sudo wget https://raw.githubusercontent.com/pfelk/pfelk/main/etc/pfelk/conf.d/45-cleanup.conf -P /etc/pfelk/conf.d/
sudo wget https://raw.githubusercontent.com/pfelk/pfelk/main/etc/pfelk/conf.d/50-outputs.conf -P /etc/pfelk/conf.d/
c5. Download pfelk.grok pattern
sudo wget https://raw.githubusercontent.com/pfelk/pfelk/main/etc/pfelk/patterns/pfelk.grok -P /etc/pfelk/patterns/ sudo wget https://raw.githubusercontent.com/pfelk/pfelk/main/etc/pfelk/patterns/openvpn.grok -P /etc/pfelk/patterns/
c6. Download pfelk databases
sudo wget https://raw.githubusercontent.com/pfelk/pfelk/main/etc/pfelk/databases/rule-names.csv -P /etc/pfelk/databases/
sudo wget https://raw.githubusercontent.com/pfelk/pfelk/main/etc/pfelk/databases/service-names-port-numbers.csv -P /etc/pfelk/databases/ sudo wget https://raw.githubusercontent.com/pfelk/pfelk/main/etc/pfelk/databases/private-hostnames.csv -P /etc/pfelk/databases
c7. Download pfELK Error Log Script (Optional)
sudo wget https://raw.githubusercontent.com/pfelk/pfelk/main/etc/pfelk/scripts/error-data.sh -P /etc/pfelk/scripts/c8. Make pfELK Error Log Script Executable (Optional)
sudo chmod +x /etc/pfelk/scripts/error-data.sh(d) - Configure Services
Elasticsearch, Kibana and Logstash
d1. Automatic Start (Start Servies on Boot)
sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service
sudo /bin/systemctl enable kibana.service
sudo /bin/systemctl enable logstash.service
d2. Manual Start (Start Servies Manually)
sudo -i service elasticsearch start
sudo -i service kibana start
sudo -i service logstash start
(e) - Finalize
pfELK
e1. Import the required templates (NOTE: Must be done prior to receiving logs. Otherwise, you'll need to rebuild your indices)
- Click ☰ in the upper left corner
- Click on Dev Tools located near the bottom under the Management heading
- Paste the contents of each file located here (repeat for each file) or the following:
- Component Templates
- pfelk-mappings-ecs must be installed first as the subsequent index patterns are dependent upon these Component Templates.
- pfelk-mappings-ecs - Must be installed first
- pfelk-ilm
- Index Templates
- Click the green triangle after pasting the contents (one at a time) into the console
- The Saved Objects may be downloaded here
- Click on the ☰ in the upper left corner
- Click on Stack Management located near the bottom under the Management heading
- Click on Saved Objects located under the Kibana heading
- Click "Import" (YouTube Tutorial Video 1 and Video 2)
- pfSense - Navigate to Status >> System Logs [Settings] and configure as depicted below:
- Enable Remote Logging
- Provide "Server 1" address (this is the IP address of the ELK installation [e.g. 192.168.1.60:5140])
- Select "Firewall events"
- OPNsense - Navigate to System >> Settings >> Logging / targets and configure as depicted below:
- The Hostname is the IP address of where you installed ELK
- The port should be set to 5140
(f) - Troubleshooting
pfELK
f1. Check Status of each process
systemctl status elasticsearch.service
systemctl status kibana.service
systemctl status logstash.service
f2. Review Logstash Logs for errors
cat /var/log/logstash/logstash-plain.log
f3. Generate pfELK log
sudo ./etc/pfELK/error-data.sh
- This will generate a log file within /etc/pfELK/logs/
- Utilize the log file to aid in troubleshooting
f4. Need additional assistance, visit pfELK wiki page via GitHub
f5. Video installation tutorial via 3ilson YouTube Channel
f6. Submit an Issues via here, leave a comment below or send an email
f7. Discuss, collaborate, troubleshoot, etc... within the pfELK community on Gitter
If this helped, feel free to make a contribution:
