FortiGate III Student Guide-Online
April 28, 2017 | Author: rickwalsh8812 | Category: N/A
Short Description
Download FortiGate III Student Guide-Online...
Description
DO NOT REPRINT © FORTINET
FortiGate III Student Guide for FortiGate 5.2.1
DO NOT REPRINT © FORTINET FortiGate III Student Guide for FortiGate 5.2.1 Last Updated: 20 July 2015 We would like to acknowledge the following major contributors: Francois Ropert, David Chan, Adrian Buckley, Ondrej Holecek, Stephane Hamelin, and Mike Lobban ®
®
®
Fortinet , FortiGate , and FortiGuard are registered trademarks of Fortinet, Inc. in the U.S. and other jurisdictions, and other Fortinet names herein may also be trademarks, registered or otherwise, of Fortinet. All other product or company names may be trademarks of their respective owners. Copyright © 2002 - 2015 Fortinet, Inc. All rights reserved. Contents and terms are subject to change by Fortinet without prior notice. No part of this publication may be reproduced in any form or by any means or used to make any derivative such as translation, transformation, or adaptation without permission from Fortinet, Inc., as stipulated by the United States Copyright Act of 1976.
DO NOT REPRINT © FORTINET Table of Contents VIRTUAL LAB BASICS ...................................................................................7 Topology..................................................................................................................................8 Logging In ...............................................................................................................................8 Disconnections/Timeouts .............................................................................................................................13
Transferring Files to the VM....................................................................................................13 Using HTML5 Instead of Java ................................................................................................13 Screen Resolution ...................................................................................................................14 International Keyboards ..........................................................................................................14 Troubleshooting Tips ..............................................................................................................15
SYSTEM RESOURCES ...................................................................................17 Objectives ...............................................................................................................................17 Time to Complete ....................................................................................................................17 System, Processes and Crashlog...........................................................................................18
NETWORK ....................................................................................................21 Objectives ...............................................................................................................................21 Time to Complete ....................................................................................................................21 Exploring the Session Table ...................................................................................................22 Traffic sniffer ...........................................................................................................................25 Break and Fix: Connectivity Issues .........................................................................................28 Tips for Troubleshooting ...............................................................................................................................28
DO NOT REPRINT © FORTINET FIREWALL POLICIES .....................................................................................30 Objectives ...............................................................................................................................30 Time to Complete ....................................................................................................................30 Traffic Shaping ........................................................................................................................31 Break and Fix: FTP Traffic ......................................................................................................32 Tips for Troubleshooting ...............................................................................................................................33
FIREWALL AUTHENTICATION .........................................................................34 Objectives ...............................................................................................................................34 Time to Complete ....................................................................................................................34 Break and Fix: LDAP Authentication ......................................................................................35 Tips for Troubleshooting ...............................................................................................................................35
FSSO ..........................................................................................................37 Objectives ...............................................................................................................................37 Time to Complete ....................................................................................................................37 Installing FSSO .......................................................................................................................38 Break and Fix: FSSO ..............................................................................................................42 Tips for Troubleshooting ...............................................................................................................................42
IPSEC ..........................................................................................................44 Objectives ...............................................................................................................................44 Time to Complete ....................................................................................................................44 Break and Fix: IPsec VPN ......................................................................................................45 Tips for Troubleshooting ...............................................................................................................................45
SECURITY PROFILES ....................................................................................47 Objectives ...............................................................................................................................47 Time to Complete ....................................................................................................................47
DO NOT REPRINT © FORTINET Break and Fix: Protection Profiles Part 1 ................................................................................48 Tips for Troubleshooting ...............................................................................................................................48
Break and Fix: Protection Profiles Part 2 ................................................................................49 Tips for Troubleshooting ...............................................................................................................................50
EXPLICIT WEB PROXY ..................................................................................51 Objectives ...............................................................................................................................51 Time to Complete ....................................................................................................................51 Break and Fix: Web Proxy ......................................................................................................52 Tips for Troubleshooting ...............................................................................................................................53
OPERATION MODES......................................................................................55 Objectives ...............................................................................................................................55 Time to Complete ....................................................................................................................55 Transparent Mode ...................................................................................................................56 NAT/Route Mode ....................................................................................................................60 Break and Fix: NAT/Route Mode ............................................................................................62 Tips for Troubleshooting ...............................................................................................................................62
EXTERNAL BGP ...........................................................................................63 Objectives ...............................................................................................................................63 Time to Complete ....................................................................................................................63 Break and Fix: BGP ................................................................................................................64 Tips for Troubleshooting ...............................................................................................................................64
OSPF ..........................................................................................................66 Objectives ...............................................................................................................................66 Time to Complete ....................................................................................................................66 Break and Fix: OSPF ..............................................................................................................67 Tips for Troubleshooting ...............................................................................................................................67
DO NOT REPRINT © FORTINET HIGH AVAILABILITY ......................................................................................69 Objectives ............................................................................................................................ 69 Time to Complete ................................................................................................................. 69 Break and Fix: High Availability ............................................................................................ 70 Tips for Troubleshooting ...............................................................................................................................71
APPENDIX A: ADDITIONAL RESOURCES........................................................72 APPENDIX B: PRESENTATION SLIDES ...........................................................73 Module 1: Troubleshooting Concepts...................................................................................74 Module 2: System Resources...............................................................................................97 Module 3: Network..............................................................................................................147 Module 4: Firewall Policies.................................................................................................174 Module 5: Firewall Authentication.......................................................................................211 Module 6: FSSO.................................................................................................................241 Module 7: IPsec..................................................................................................................275 Module 8: Security Profiles.................................................................................................312 Module 9: Explicit Web Proxy.............................................................................................368 Module 10: Operation Modes.............................................................................................390 Module 11: External BGP...................................................................................................424 Module 12: OSPF...............................................................................................................456 Module 13: High Availability...............................................................................................496
DO NOT REPRINT © FORTINET
Virtual Lab Basics
Virtual Lab Basics In this class, you will use a virtual lab for hands-on exercises. This section explains how to connect to the lab and its virtual machines. It also shows the topology of the virtual machines in the lab. Note: If your trainer asks you to use a different lab, such as devices physically located in your classroom, please ignore this section. This applies only to the virtual lab accessed through the Internet. If you do not know which lab to use, please ask your trainer.
FortiGate III Student Guide
7
DO NOT REPRINT © FORTINET
Virtual Lab Basics
Topology FORTIMANAGER port2 10.200.1.241 port1 10.0.1.241
10.200.1.1/24 port1
10.200.1.254 eth1
LINUX
10.200.3.254 eth3
STUDENT port3 10.0.1.254/24
FortiGate
port2 10.200.2.1/24
10.200.3.1/24 port4 REMOTE
eth2 10.200.2.254
port6 10.0.2.254/24
FortiGate
eth4 10.200.4.254
port5 10.200.4.1/24
Internet WIN-STUDENT 10.0.1.10
WIN-REMOTE 10.0.2.10
Logging In 1. Run the System Checker. This will fully verify both:
compatibility with the virtual lab environment's software, and that your computer can connect
It can also diagnose problems with your Java Virtual Machine, firewall, or web proxy. Use the URL for your location. North America/South America: https://remotelabs.training.fortinet.com/training/syscheck/?location=NAM-West Europe/Middle East/Africa: https://remotelabs.training.fortinet.com/training/syscheck/?location=Europe Asia/Pacific: https://remotelabs.training.fortinet.com/training/syscheck/?location=APAC If a security confirmation dialog appears, click Run.
FortiGate III Student Guide
8
DO NOT REPRINT © FORTINET
Virtual Lab Basics
If your computer successfully connects to the virtual lab, the result messages for the browser and network checks will each display a check mark icon. Continue to the next step. If a browser test fails, this will affect your ability to access the virtual lab environment. If a network test fails, this will affect the usability of the virtual lab environment. For solutions, either click the Support Knowledge Base link or ask your trainer. 2. With the user name and password from your trainer, log into the URL for the virtual lab. Either: https://remotelabs.training.fortinet.com/
FortiGate III Student Guide
9
DO NOT REPRINT © FORTINET
Virtual Lab Basics
https://virtual.mclabs.com/
3. If prompted, select the time zone for your location, and then click Update. This ensures that your class schedule is accurate. 4. Click Enter Lab.
A list of virtual machines that exist in your virtual lab should appear. From this page, you can access the console of any of your virtual devices by either:
clicking on the device’s square, or selecting System > Open.
FortiGate III Student Guide
10
DO NOT REPRINT © FORTINET
FortiGate III Student Guide
Virtual Lab Basics
11
DO NOT REPRINT © FORTINET
Virtual Lab Basics
5. Click Win-Student to open a connection to that server.
A new window should open within a few seconds. (Depending on your account’s preferences, the window may be a Java applet. If this fails, you may need change browser settings to allow Java to run on this web site. You also may need to review and accept an SSL certificate.)
Depending on the virtual machine, the applet provides access to either the GUI or a textbased CLI. Connections to Windows machines will use a Remote Desktop-like GUI. The applet should automatically log in, then display the Windows desktop. For most lab exercises, you will connect to this VM. FortiGate III Student Guide
12
DO NOT REPRINT © FORTINET
Virtual Lab Basics
Disconnections/Timeouts If your computer’s connection with the virtual machine times out or if you are accidentally disconnected, to regain access, return to the initial window/tab that contains your session’s list of VMs and open the VM again. If your session frequently times out or does not connect, ask your instructor.
Transferring Files to the VM When using the Java applet to connect to a VM, you can drag-and-drop files from your computer to the VM. For example, if you have a FortiGate configuration file that you want to upload to your lab VM, you could create it on your computer, and then drag it into the Java application window that is connected to the Windows VM. Usually the destination folder is C:\Uploads. Alternatively, if you store files in a cloud service such as Dropbox or SugarSync, you can use the web browser to download them to your VM instead.
Using HTML5 Instead of Java When you open a VM, your browser may download and use a Java application to connect to the virtual lab’s VM. This means that Java must be installed, updated, and enabled in your browser. Alternatively, you can use HTML5 instead. Click the Settings button, and then select Use Java Client. Click Save & Disconnect, then log in again. (To use this preference, your browser must allow cookies.)
FortiGate III Student Guide
13
DO NOT REPRINT © FORTINET
Virtual Lab Basics
When connecting to a VM, your browser should then open a display in a new window or tab.
Screen Resolution Some Fortinet devices' user interfaces require a minimum screen size. In the Java client, to configure the screen resolution, click the arrow at the top of the window.
In the HTML 5 client, to configure screen resolution, open the System menu.
International Keyboards If characters in your language don’t display correctly, keyboard mappings may not be correct.
FortiGate III Student Guide
14
DO NOT REPRINT © FORTINET
Virtual Lab Basics
To solve this in the HTML 5 client, open the Keyboard menu at the top of the window. Choose to either display an on-screen keyboard, or send text from your computer to the VM's clipboard.
To solve this in the Java client, copy and paste between your computer and the Java applet. This sends special characters or combinations using the keyboard icon at the top of the applet window.
Troubleshooting Tips
If the HTML 5 client does not work, try the Java client instead. Remembering this preference requires that your browser allow cookies.
Do not connect to the virtual lab environment through a low-bandwidth or high-latency connection, including VPN tunnels or wireless such as 3G or Wi-Fi. For best performance, use a stable broadband connection such as a LAN.
Do not disable or block Java applets. On Mac OS X since early 2014, to improve security, Java has been disabled by default. In your browser, you must allow Java for this web site. On Windows, if the Java applet is allowed and successfully downloads, but does not appear to launch, you can open the Java console while troubleshooting. To do this, open the Control Panel, click Java, and change the Java console setting to be Show console. Network firewalls can also block Java executables. Note: JavaScript is not the same as Java.
FortiGate III Student Guide
15
DO NOT REPRINT © FORTINET
Virtual Lab Basics
Prepare your computer's settings: o
Disable screen savers
o
Change the power saving scheme so that your computer is always on, and does not go to sleep or hibernate
If disconnected unexpectedly from any of the virtual machines (or from the virtual lab portal), please attempt to reconnect. If unable to reconnect, please notify the instructor.
If during the labs, particularly when reloading configuration files, you see a message similar to the one shown below, the VM is waiting for a response to the authentication server.
To retry immediately, go to the console and enter the CLI command: exec update-now
FortiGate III Student Guide
16
DO NOT REPRINT © FORTINET
System Resources
System Resources During this lab, you will learn to use some system and memory debug commands to describe the status of the unit. Additional, you will generate and analyze a crashlog entry after intentionally killing one of the FortiGate processes.
Objectives
Use debug commands to diagnose system problems
Use the crashlog for diagnostics
Time to Complete Estimated: 15 minutes
FortiGate III Student Guide
17
DO NOT REPRINT © FORTINET
System Resources
System, Processes and Crashlog 1. From the Win-Student server, log on to the Student FortiGate’s GUI using the account admin with no password: http://10.0.1.254 2. From System -> Dashboard -> Status click the Restore link inside the System Information widget:
3. Click Browse and select the configuration file for this lab: Resources\FortiGate III\System\Student\student-system.conf Click Restore. The Student FortiGate will reboot. Wait a few minutes until it is back up. 4. Using PuTTY, connect SSH to the Student FortiGate CLI (use the account admin with no password) and execute these commands to check the memory usage: # get system status # get system performance status # diagnose hardware sysinfo memory # diagnose hardware sysinfo shm Analyze the outputs from the above commands and answer these questions:
Is this unit running a 32-bit or 64-bit FortiOS? Does it have a hard disk for logging? How much memory is available? Is the unit in conserve mode? Why are the total high memory (HighTotal) and available high memory (HighFree) 0 MB?
FortiGate III Student Guide
18
DO NOT REPRINT © FORTINET
System Resources
5. Execute now the following command to display the top 50 processes: # diagnose sys top 6. Try to find one of these three processes: reportd, miglogd, or ipshelper. Write down its process ID (the first number from left to right):
7. Use the following command to "kill" the chosen process: # diagnose sys kill 11 11 is the kill signal. In this case the FortiGate kills the process by sending a segmentation fault (number 11) signal. Caution: We use the kill command in this exercise to reproduce a process failure. Be careful although when doing it in a FortiGate that is in production. Improperly killing a process might make a FortiGate system unstable. 8. Execute the following command one more time: # diagnose sys top Observe that the killed process is running again, but this time it is using a higher ID number. Each time a process starts, it uses the next available process ID number. 9. Now, check the crashlog: # diagnose debug crashlog read The output should contain some entries similar to these ones: 93: 2015-03-04 07:47:34 Signal was sent to process by user 94: 2015-03-04 07:47:34 firmware FortiGate-VM64 v5.2.1,build0618b618,140915 (GA) (Release) 95: 2015-03-04 07:47:34 application reportd
FortiGate III Student Guide
19
DO NOT REPRINT © FORTINET
System Resources
96: 2015-03-04 07:47:34 *** signal 11 (Segmentation fault) received *** 97: 2015-03-04 07:47:34 Register dump: 98: 2015-03-04 07:47:34 RAX: fffffffffffffffc 0000000000000000
RBX:
99: 2015-03-04 07:47:34 RCX: ffffffffffffffff 0000000000000400
RDX:
100: 2015-03-04 07:47:34 R8: 0000002a95d49de0
0000000000000000
R9:
... 120: 2015-03-04 07:47:35 [0x0043d14f] => /bin/reportd 121: 2015-03-04 07:47:35 [0x0043abfa] => /bin/reportd 122: 2015-03-04 07:47:35 [0x2a95c40475] => ../lib/libc.so.6 (__libc_start_main+0x000000f5) 123: 2015-03-04 07:47:35 liboffset 00021475 124: 2015-03-04 07:47:35 [0x0043aca1] => /bin/reportd 125: 2015-03-04 07:47:35 reportd received a signal - 11 126: 2015-03-04 07:47:36 the killed daemon is /bin/reportd: status=0x0 Check the first three lines. They contain the FortiOS build number, the name of the process that failed (or was killed) and the kill signal number.
FortiGate III Student Guide
20
DO NOT REPRINT © FORTINET
Network
Network The following lab exercises show how to use some debug commands to troubleshoot connectivity problems. You will analyze the information in the FortiGate session table, run the built-in sniffer and use the debug flow to understand how the FortiGate is processing each IP packet.
Objectives
Analyze the information in the session table
Capture traffic using the built-in sniffer tool
Use some CLI troubleshooting utilities and tools
Time to Complete Estimated: 50 minutes
FortiGate III Student Guide
21
DO NOT REPRINT © FORTINET
Network
Exploring the Session Table During this exercise you will analyze the information displayed in the FortiGate session table. 1. From the Win-Student server, log on to the Remote FortiGate’s GUI first using the account admin with no password: http://10.200.3.1 2. Find the Resource folder on the desktop and upload the Remote configuration file for this lab: Resources\FortiGate III\General\Remote\remote-general.conf The Remote FortiGate will reboot. Wait a few minutes until it is back up. 3. Log on to the Student FortiGate’s GUI using the account admin with no password: http://10.0.1.254 4. Upload the Student configuration file for this lab: Resources\FortiGate III\General\Student\student-general.conf The Student FortiGate will reboot. Wait a few minutes until it is back up. 5. Open a command prompt window in the Win-Student server and execute a ping to the Student FortiGate's default gateway: > ping 10.200.1.254 6. Using PuTTY, connect SSH to the Student FortiGate CLI and execute these commands: # diagnose sys session filter clear # diagnose sys session filter proto 1 # diagnose sys session filter dst 10.200.1.254 # diagnose sys session list Analyze the information related with the ICMP session created for the test traffic: session info: proto=1 proto_state=00 duration=726 expire=63276 timeout=64000 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=3 origin-shaper= reply-shaper= per_ip_shaper= ha_id=0 policy_dir=0 tunnel=/ state= may_dirty none app_ntf
FortiGate III Student Guide
22
DO NOT REPRINT © FORTINET
Network
statistic(bytes/packets/allow_err): org=240/4/1 reply=240/4/1 tuples=2 orgin->sink: org pre->post, reply pre->post dev=4->2/2->4 gwy=10.200.1.254/10.0.1.10 hook=post dir=org act=snat 10.0.1.10:1>10.200.1.254:8(10.200.1.1:62464) hook=pre dir=reply act=dnat 10.200.1.254:62464>10.200.1.1:0(10.0.1.10:1) misc=0 policy_id=1 auth_info=0 chk_client_info=0 vd=0 serial=00000243 tos=ff/ff ips_view=0 app_list=0 app=0 dd_type=0 dd_mode=0 Observe the following from the session:
The may_dirty flag The line containing statistics, which should correctly display the amount of ICMP packets sent and received The source NAT information The ID of the policy matching the traffic The protocol state, whose value is always 00 for the case of ICMP traffic
You will also notice that the expiration timer (expire) and timeout are unusually high. The default timeout for ICMP sessions is 60 seconds. For the purpose of giving more time to analyze the session information, the ICMP session timeout was increased to 64.000 seconds. You can see this configuration by typing the CLI command: # show system session-ttl 7. Stop the ping (if it is still running) and access the Student FortiGate GUI. Go to Policy & Objects > Policy > IPv4 and edit the firewall policy with the sequence number 1 (the first one from top to bottom). Click Enable this policy and then OK. After a firewall policy configuration change, the FortiGate adds the dirty flag to all the session with the may_dirty flag. Next time there is traffic matching any of those sessions, the FortiGate will re-evaluate the action to take. 8. Execute this command in the Student FortiGate CLI and observe that the session has the dirty flag now: # diagnose sys session list 9. Run the ping one more time from the Win-Student server to 10.200.1.254: > ping 10.200.1.254 It should fail as the firewall policy enabled earlier is blocking ICMP traffic. Check quickly the session information one more time: # diagnose sys session list If you do it fast enough, you will notice that the session is still there but the block flag was added. All traffic matching a session with that flag is denied. Also, the session expiration
FortiGate III Student Guide
23
DO NOT REPRINT © FORTINET
Network
time is much smaller now. The session will remain in the FortiGate memory until this timer expires (30 seconds). 10. Before proceeding to the next lab exercise, go to Policy & Objects > Policy > IPv4 and disable the firewall policy with the sequence number 1 (the one blocking the ICMP traffic.)
FortiGate III Student Guide
24
DO NOT REPRINT © FORTINET
Network
Traffic sniffer During this exercise, you will use the FortiGate built-in sniffer to capture traffic. After that, you will use a Perl script to convert the capture to a PCAP file that can be analyzed by a packet analyzer, such as Wireshark. 1. Open a SSH connection to the Student FortiGate using PuTTY. 2. Click on the upper left icon and select Change Settings:
Go to Session -> Logging and select All session output. Then click Browse and select the folder c:\Perl64\bin. Click Save, and then Apply. With this change, PuTTY will save all the sniffer output into a text file name putty.log:
FortiGate III Student Guide
25
DO NOT REPRINT © FORTINET
Network
3. Type the following command in the Student FortiGate CLI to start the sniffer: # diagnose sniffer packet port1 "host 10.200.1.254 and port 80" 3 4. Open a browser and access this URL: http://10.200.1.254 You should observe the packets captured in the PuTTY window. 5. Close PuTTY and open a command prompt window. Execute these commands: > cd \Perl64\bin > perl fgt2eth.pl –in putty.log The Perl script fgt2eth.pl converts the output captured to a PCAP file with the name putty.log.pcap. 6. Use Windows File Explorer and double click the created file: c:\Perl64\bin\putty.log.pcap This starts Wireshark and opens the file for analysis. 7. Observe the information in the packets captured. Right click any packet and select Follow TCP Stream:
FortiGate III Student Guide
26
DO NOT REPRINT © FORTINET
Network
Observe the new Window that pops up. It shows the application-layer data between the client and the server for that specific TCP session:
Note: Follow TCP Stream is a useful tool to troubleshoot problems at the application layer.
FortiGate III Student Guide
27
DO NOT REPRINT © FORTINET
Network
Break and Fix: Connectivity Issues In this exercise, your environment is simulating the following customer network:
10.200.1.254/24 STUDENT FortiGate
10.200.1.1/24 port1
Web server 10.200.3.254
port3 10.0.1.254/24
port2 10.200.2.1/24 10.200.2.10/24 WIN-STUDENT 10.0.1.10
REMOTE HOST 10.200.4.1/24
There are however four problems: 1. Although the Telnet protocol is enabled for administrative access in the Student FortiGate's port3 (10.0.1.254), you cannot access the unit's CLI using telnet. 2. You cannot access the web server (http://10.200.3.254) from Win-Student. 3. You cannot ping the remote host (10.200.4.1) from Win-Student. 4. You cannot access the GUI of the router 10.200.1.254 from Win-Student. The router GUI must be accessible by using the URL: http://10.200.1.254:88 Find the causes of these problems by using first debug commands, before looking for configuration mistakes. In which of the four problems the FortiGate is doing something wrong and in which ones it is not?
Tips for Troubleshooting
Can you ping the destination IP address from the Win-Student server?
Use the sniffer tool to verify that the traffic is actually arriving to the FortiGate's port3. Use verbosity 4 or 6 and a filter that can capture the traffic both ways
If the traffic is not intended to terminate in the FortiGate, use the sniffer to check that it is actually been forwarded to the next hop IP address (use the network diagram provided.) Again, use a filter in the sniffer that can capture the traffic both ways
FortiGate III Student Guide
28
DO NOT REPRINT © FORTINET
Network
Check the session table. Is the FortiGate creating the session? Check the session protocol state. Do you see anything wrong there?
Clear the related session (if any) from the session table, enable the debug flow and generate more test traffic. Do you see any debug flow error?
Try to ping the next hop IP address from the Student FortiGate. Sniffer the traffic to the next hop IP address while doing it. You can have two simultaneous SSH connections, one running the sniffer and another one running the ping
FortiGate III Student Guide
29
DO NOT REPRINT © FORTINET
Firewall Policies
Firewall Policies During these lab exercises you will configure and monitor traffic shaping. Additionally, you will troubleshoot and fix a connection problem to a FTP server.
Objectives
Monitor statistics related with traffic shaping
Troubleshoot a FTP connection issue
Time to Complete Estimated: 40 minutes
FortiGate III Student Guide
30
DO NOT REPRINT © FORTINET
Firewall Policies
Traffic Shaping 1. Log on to the Student FortiGate’s GUI using the account admin with no password: http://10.0.1.254 2. Upload the Student configuration file for this lab: Resources\FortiGate III\Firewall-Policies\Student\student-policy.conf The Student FortiGate will reboot. Wait a few minutes until it is back up. 3. Go to Policy & Objects > Objects > Traffic Shapers and click Create New. Configure the following settings: Type
Shared
Name
SharedPolicy
Apply shaper
All policies using this shaper
Traffic Priority
High
Max Bandwidth
10 Kb/s
Click OK. 4. Go to Policy & Objects > Policy > IPv4 and edit the first policy on the top. Enable Shared Shaper and select SharedPolicy. Click OK. 5. From a browser in Win-Student go to http://www.youtube.com and play some videos. 6. While plying the videos, execute these CLI commands: # diagnose firewall shaper traffic-shaper stats # diagnose firewall shaper traffic-shaper list Locate the counters for packet drops. Execute the above commands a few times more and notice how those counters increase with the traffic. 7. Before proceeding to the next lab exercise, go to Policy & Objects > Policy > IPv4 and edit the first policy on the top one more time. Disable Shared Shaper and click OK.
FortiGate III Student Guide
31
DO NOT REPRINT © FORTINET
Firewall Policies
Break and Fix: FTP Traffic A FTP server is running in the Linux server 10.200.3.254:222. The network administrator has installed FileZilla in all the workstations. The administrator has also added a pre-configured Site profile to FileZilla called FTPsite. To connect to the server from any workstation, users open FileZilla, click Site Manager, select the site FTPsite and click Connect:
However, you cannot connect to the FTP server from Win-Student. FileZilla shows this error after each connection attempt:
The problem only happens with the workstations connected behind the Student FortiGate. Workstations in other subnets can connect successfully. FortiGate III Student Guide
32
DO NOT REPRINT © FORTINET
Firewall Policies
Can you find out what the FortiGate is doing wrong? What has to be done to fix the problem?
Tips for Troubleshooting
Understand first which TCP ports are used for this connection. The control channel is using port TCP 222. The data channel is using the standard port TCP 20
Understand also how the traffic flows. Is this a FTP server working in active or passive mode? In active mode the data channel is initiated by the server. In passive mode the data channel is initiated by the client. Sniffer the traffic in the FortiGate and Linux server to determine who is initiating the data channel. To run the sniffer in the Linux server follow these steps: 1. Connect SSH to the Linux server (10.200.1.254). Use the username root with the password password 2. Execute the following command to sniffer the data channel: # tcpdump -v -i any -nn port 20 You can also sniffer the control channel traffic with this other command: # tcpdump -v -i any -nn port 222
Use the FortiGate's built-in sniffer to capture the control channel traffic (port 222) before and after the FortiGate. Use a verbosity level of either 3 or 6 and save the output to a file. After that, use the Perl script to convert it to Wireshark (as explained in an earlier lab exercise) and analyze it
Run the debug flow over the FTP control channel and analyze the output. Is there anything missing there?
FortiGate III Student Guide
33
DO NOT REPRINT © FORTINET
Firewall Authentication
Firewall Authentication During this lab you will learn to use the authentication and LDAP debug commands to troubleshoot an authentication issue.
Objectives
Monitor the status of authenticated users
Troubleshoot problems related with LDAP authentication
Time to Complete Estimated: 40 minutes
FortiGate III Student Guide
34
DO NOT REPRINT © FORTINET
Firewall Authentication
Break and Fix: LDAP Authentication 1. Log on to the Student FortiGate’s GUI using the account admin with no password: http://10.0.1.254 2. Upload the Student configuration file for this lab: Resources\FortiGate III\Firewall-Authentication\Student\studentauthentication.conf The Student FortiGate will reboot. Wait a few minutes until it is back up. An administrator has configured the Student FortiGate to do LDAP authentication against the Windows AD server located at 10.0.1.10 (Win-Student). However, authentication is failing. Two LDAP users have been created in the Windows AD Server:
Username: student, password: Fort1net Must not have access to information technology sites, such as www.fortinet.com Belongs to the Windows AD group: CN=Domain Users,CN=Users,DC=trainingAD,DC=training,DC=lab Traffic from this user must match the firewall policy crated for the user group LDAPUsers, which contains the web filter profile NoITSites. Do not change this web filtering configuration.
Username: administrator, password: password Must have unrestricted access to the Internet Belongs to the Windows AD group: CN=Enterprise Admins,CN=Users,DC=trainingAD,DC=training,DC=lab Traffic from this user must match the firewall policy created for the user group Enterprise Admins, which does not have any web filter profile. Do not create any web filter profile for this policy. Leave it without any.
Use the authentication and LDAP debug commands learned to isolate and fix the problem. Can you explain why the FortiGate is not challenging users to authenticate? Can you change the FortiGate configuration to fix the problem? Can you change the FortiGate configuration to properly restrict the Internet access to the user student, while leaving unrestricted access to the user administrator?
Tips for Troubleshooting
First, test the LDAP authentication from the CLI after enabling the real time debug command: diagnose debug application fnbamd -1 diagnose debug enable
FortiGate III Student Guide
35
DO NOT REPRINT © FORTINET
Firewall Authentication
diagnose test authserver ldap WindowsLDAP administrator password diagnose test authserver ldap WindowsLDAP student Fort1net
Check the Distinguished Name (DN) for student and administrator, by running these commands in Win-Student: dsquery user -name student dsquery user -name administrator
Once the LDAP CLI test works, check the firewall authentication by browsing the Internet from Win-Student. Look at the session table or run the debug flow to know which firewall policy is matching the traffic
The output of the LDAP test command shows the user groups for each user. Compare them with the groups configured in each firewall policy
After any configuration change, de-authenticate the users from the FortiGate and clear the browser cache (or refresh the page with the F5 key). It is also recommended to clear the related entries in the session table: # diagnose sys session filter dport 80 # diagnose sys session clear To de-authenticate a user, go to User & Device -> Monitor -> Firewall, select the user and click on De-authenticate
FortiGate III Student Guide
36
DO NOT REPRINT © FORTINET
FSSO
FSSO During this lab you will install the FSSO collector agent and troubleshoot a FSSO problem.
Objectives
Check the connectivity between the FortiGate and the CA
Track user logon events in the DC, CA and FortiGate
List the active FSSO users
Troubleshoot a FSSO problem
Time to Complete Estimated: 40 minutes
FortiGate III Student Guide
37
DO NOT REPRINT © FORTINET
FSSO
Installing FSSO 1. Log on to the Student FortiGate’s GUI using the account admin with no password: http://10.0.1.254 2. Upload the Student configuration file for this lab: Resources\FortiGate III\FSSO\Student\student-FSSO.conf The Student FortiGate will reboot. Wait a few minutes until it is back up. 3. On Win-Student, right-click the Fortinet Single Sign On (FSSO) installation file located in Resources\FSSO, then select Run as administrator. This should launch the Fortinet Single Sign On Agent Installation Wizard. Follow the wizard to install the agent on Win-Student. 4. When prompted for the Windows server administrator password, enter "password":
Click Next. 5. In the Install Options window, accept the default settings:
Click Next. 6. Click Install to complete the installation. FortiGate III Student Guide
38
DO NOT REPRINT © FORTINET
FSSO
7. At the end of the Single Sign On Agent installation, the Launch DC Agent Install Wizard option will be selected. Click Finish to complete the collector agent Installation. This launches the Domain Controller Agent Installation Wizard. 8. In the Install DC Agent Wizard, accept the Collector Agent IP Address of 10.0.1.10 and the Collector Agent Listening Port of 8002.
Click Next. 9. Select the TRAININGAD:trainingAD.training.lab domain to monitor. Click Next. 10. Only the student account needs to be monitored in this exercise. Expand the TRAININGAD domain and disable all the users in the TRAININGAD domain EXCEPT for student:
Click Next. 11. Set the Working Mode to Polling Mode and select Check Windows Security Event Logs.
FortiGate III Student Guide
39
DO NOT REPRINT © FORTINET
FSSO
Click Next. 12. After the installation, open the Windows start screen and run the application Configure Fortinet Single Sign-on. Perform the following tasks in the Fortinet single sign-on agent configuration window:
Change the Password to Fortinet. Change the Workstation verify interval to 0 Change the Log level to Information Enable Log logon events in separate logs
Click Apply. 13. Click Show Monitored DCs to verify the communication between the collector agent and the domain controller agent. The IP address of 10.0.1.10 should show as being logged in. Click Close. 14. Click Select Domains to Monitor and verify that the TRAININGAD:trainingAD.training.lab domain is selected. Click OK.
FortiGate III Student Guide
40
DO NOT REPRINT © FORTINET
FSSO
15. Click Set Group Filters. Click Add and enable the Default filter. Click Advanced and expand the domain name of TRAININGAD. From the expanded list select Users and Domain Admins. Click Add, then OK.
Click OK. Click Save & Close to close the Fortinet single sign-on agent configuration window.
FortiGate III Student Guide
41
DO NOT REPRINT © FORTINET
FSSO
Break and Fix: FSSO In this network the collector agent has been installed in Win-Student. An administrator has configured the Student FortiGate to allow Internet access only to active FSSO users. However, it is not working as desired. Active FSSO users do not have Internet access. Use the authentication and FSSO debug commands learned to isolate and fix the problem. To test the FSSO authentication, generate first a login event following these steps: 1. On Win-Student, run the Windows Remote Desktop Connections application. 2. Enter the computer IP address 10.0.1.10:
Log in with these credentials: Username:
Student
Password:
Fort1net
Ignore the error message indicating that the user is not authorized for remote login. The objective of these steps is just to generate a logon event without rebooting the server. 3. After that, test the Internet access from a browser. Can you explain why the FortiGate is blocking the traffic? Can you change the FortiGate or/and collector agent configurations to fix the problem?
Tips for Troubleshooting
Check the active FSSO users in the collector agent by clicking Show logon users
Use the following command to check the active FSSO users in the FortiGate: # diagnose debug authd fsso list
Use the FortiGate real time debug command for FSSO: # diagnose debug application authd 8256 # diagnose debug enabled
Check the collector agent logs
FortiGate III Student Guide
42
DO NOT REPRINT © FORTINET
FSSO
Use the Windows Remote Desktop Connections application after each configuration change to generate new login events
FortiGate III Student Guide
43
DO NOT REPRINT © FORTINET
IPsec
IPsec During this lab you will troubleshoot an IPsec VPN problem.
Objectives
Use the IKE real time debug to isolate problems during the phase 1 and phase 2 negotiations
Use the debug flow tool to isolate IPsec traffic flow issues
Monitor the status of an IPsec VPN
Time to Complete Estimated: 90 minutes
FortiGate III Student Guide
44
DO NOT REPRINT © FORTINET
IPsec
Break and Fix: IPsec VPN 1. From the Win-Student server, log on to the Student FortiGate’s GUI using the account admin with no password: http://10.0.1.254 2. Upload the Student configuration file for this lab: Resources\FortiGate III\VPN\Student\student-vpn.conf The Student FortiGate will reboot. Wait a few minutes until it is back up. 3. Log on to the Remote FortiGate’s GUI first using the account admin with no password: http://10.200.3.1 Upload the Remote configuration file for this lab: Resources\FortiGate III\VPN\Remote\remote-vpn.conf The Remote FortiGate will reboot and load the new configuration. An administrator has created an IPsec VPN between the Student FortiGate and the Remote FortiGate. The objective is to encrypt the traffic both ways between the subnets 10.0.1.0/24 and 10.0.2.0/24. For the purpose of this lab, assume that the IP address of the Remote FortiGate will be changing frequently, so the administrator has configured the VPN in the Student FortiGate side as Dialup User. The name of the VPN in the Student side is RemoteSite. The name of the VPN in the Remote side is ToHub. There is another VPN created (for a different purpose) in the Student FortiGate with the name DialUpUsers. The VPN IPsec between Student and Remote is down. Your objective is to fix the problem, so that the tunnel comes up and you can ping from Win-Student to Win-Remote. Use the IPsec debug commands learned in this lesson to isolate and fix the problem. The solution requires:
Keeping the Remote Gateway type in the VPN RemoteSite as Dialup User (for the reason explained before)
No configuration changes in the DialUPUsers VPN in the Student FortiGate, as this VPN is already operative and working as expected (you do not need to test this VPN, assume that it is working)
Tips for Troubleshooting
Check first why the tunnel is not coming up, use the IKE real time debug in both sides to troubleshoot the problem: # diagnose debug application ike -1 # diagnose debug enable
After the tunnel is established, check that you can ping from Win-Student to Win-Remote. If
FortiGate III Student Guide
45
DO NOT REPRINT © FORTINET
IPsec
there is a problem, sniffer the traffic and use the debug flow
FortiGate III Student Guide
46
DO NOT REPRINT © FORTINET
Security Profiles
Security Profiles During the following exercises you will use debug commands to fix FortiGuard and web filtering issues.
Objectives
Troubleshoot FortiGuard problems
Troubleshoot web filtering problems
Fix certificate warnings during full SSL inspection
Investigate virus infections
Time to Complete Estimated: 45 minutes
FortiGate III Student Guide
47
DO NOT REPRINT © FORTINET
Security Profiles
Break and Fix: Protection Profiles Part 1 1. Log on to the Student FortiGate’s GUI using the account admin with no password: http://10.0.1.254 2. Upload the Student configuration file for this lab: Resources\FortiGate III\UTM\Student\student-UTM-1.conf The Student FortiGate will reboot. Wait a few minutes until it is back up. 3. The configuration contains two VDOMs. Go to Virtual Domains -> root -> Policy & Objects -> Policy -> IPv4. Check the firewall policy from port3 to port1. It has antivirus and web filtering enabled. 4. Then go to Virtual Domains -> root -> Policy & Objects -> Security Profiles -> Web Filter and review the profile WebFilterUsers. Some categories, such as malicious websites, streaming media, hacking and proxy avoidance are being blocked. Open a browser in Win-Student and go to these restricted web sites: http://www.youtube.com http://www.proxyavoidance.com The FortiGate is not blocking the access to those sites. Indeed, web filter does not seem to be working at all. Why isn't the FortiGate blocking the access to any restricted web site? Can you change the FortiGate configuration to fix the problem? Note: Your lab environment uses a FortiManager as a local FDS server. It contains a local copy of the FDS web rating database. The FortiGate devices validate their VM licenses against the FortiManager. They also send the rating requests to the FortiManager IP address (10.0.1.241) instead of the public FDS servers. Do not change this configuration, as it will affect the FortiGate license status.
Tips for Troubleshooting
Use the web filtering real time debug: # diagnose debug application urlfilter -1 # diagnose debug enable
Use the FortiGuard real time debug: # diagnose debug application update -1 # diagnose debug enable
FortiGate III Student Guide
48
DO NOT REPRINT © FORTINET
Security Profiles
Break and Fix: Protection Profiles Part 2 1. Log on to the Student FortiGate’s GUI using the account admin with no password: http://10.0.1.254 2. Upload the Student configuration file for this lab: Resources\FortiGate III\UTM\Student\student-UTM-2.conf The Student FortiGate will reboot. Wait a few minutes until it is back up. This configuration is similar to the previous one but it contains the fix to the problem that was troubleshot during the first exercise of this lab. The configuration also includes a web filter profile to block, among others, the following FortiGuard categories:
Proxy Avoidance Streaming Media and Download Hacking
All the restricted sites seem to be properly blocked now, such as: http://www.youtube.com (Streaming Media and Download) http://www.elite-hackers.com (Hacking) http://www.proxyavoidance.net (Proxy Avoidance) However, the administrator complains that the following two sites should be blocked, and they are not. According to him, they belong to blocked categories: http://www.metacafe.com http://www.eicar.org Additionally, customers are reporting two more problems:
They receive certificate warnings each time they connect to an HTTPS site Even though antivirus is enabled, they can still download the virus sample eicar.com located at the ftp server 10.200.3.254:222. To test it, open FileZilla and connect to the preconfigured site FTPSite. Select the Desktop as the local site folder and pub as the remote site folder. Right click the eicar.com file and select Download:
FortiGate III Student Guide
49
DO NOT REPRINT © FORTINET
Security Profiles
Why are those two sites reported by the administrator not being blocked? How can you change the FortiGate configuration to fix it? Why are users getting SSL certificate warnings? How can you resolve it? Why isn't FortiGate detecting the EICAR virus?
Tips for Troubleshooting For the web filtering problem:
Enable the following real time debug and attempt to browse the two websites not being blocked: # diagnose debug application urlfilter -1 # diagnose debug enable The output can be verbose, so save it from PuTTY to a local file.
Remember to clear the browse cache and FortiGate session after doing any configuration change
For the antivirus problem:
Sniffer the FTP traffic and analyze the output of the debug flow
Check the entry in the FortiGate session table for the FTP session
FortiGate III Student Guide
50
DO NOT REPRINT © FORTINET
Explicit Web Proxy
Explicit Web Proxy During this lab you will troubleshoot some explicit web proxy problems.
Objectives
Monitor web proxy traffic and sessions
Monitor web proxy DNS traffic
Use the web proxy real time debug
Time to Complete Estimated: 40 minutes
FortiGate III Student Guide
51
DO NOT REPRINT © FORTINET
Explicit Web Proxy
Break and Fix: Web Proxy 1. Log on to the Student FortiGate’s GUI using the account admin with no password: http://10.0.1.254 2. Upload the Student configuration file for this lab: Resources\FortiGate III\Web-Proxy\Student\student-web-proxy.conf The Student FortiGate will reboot. Wait a few minutes until it is back up. 3. Open Firefox and click Open menu. Then click Options:
4. Go to Advanced -> Network and click Settings:
FortiGate III Student Guide
52
DO NOT REPRINT © FORTINET
Explicit Web Proxy
5. Select Automatic proxy configuration URL and type the following URL: http://10.0.1.254:8080/proxy.pac 6. Restart the browser. Test the proxy by accessing any web site. Additionally, access to the Fortinet web site is essential for users. So, test it using the following URL: http://www.fortinet.com Why isn't the web proxy working at all? Can you change the FortiGate configuration to fix the problem? After fixing the web proxy, test the access to the Fortinet web site. Why isn't working yet? Can you also fix it?
Tips for Troubleshooting
Sniffer the traffic in port 8080 (web proxy traffic)
Sniffer the traffic coming from the browser: # diagnose sniffer packet any 'host 10.0.1.254 and not port 22 and not port 443' 4
Sniffer the traffic going to the web proxy IP address: # diagnose sniffer packet any 'host 10.0.1.10 and not port 22 and not port 443' 4
Use the following debug commands to check the status of the web proxy connections: # diagnose wad session list # diagnose test application wad 2200 # diagnose test application wad 110 # diagnose test application wad 104
Run the web proxy real time debug using the filter below: # config web-proxy debug-url edit fortinet set url-pattern www.fortinet.com set status enable set exact enable next edit fortiguard
FortiGate III Student Guide
53
DO NOT REPRINT © FORTINET
Explicit Web Proxy
set url-pattern www.fortiguard.com set status enable set exact enable next end # diagnose wad debug-url enable # diagnose wad console-log enable # diagnose debug enable After that, try to browse these two web sites and compare the results: http://www.fortinet.com http://www.fortiguard.com
Remember to restart the browser after any change to the PAC file
FortiGate III Student Guide
54
DO NOT REPRINT © FORTINET
Operation Modes
Operation Modes This lab has 3 exercises. The first exercise includes a FortiGate in transparent mode. During exercises 2 and 3 you will troubleshoot routing problems with two FortiGate devices in NAT/route mode.
Objectives
Describe how FortiGate routes traffic
Diagnose routing problems due to reverse path forwarding check
Identify the existing sessions that will be routed through a different path after a change in the routing table
Use debug commands to troubleshoot routing problems
Segment a layer-2 network into different broadcast domains using a FortiGate in transparent mode
Time to Complete Estimated: 45 minutes
FortiGate III Student Guide
55
DO NOT REPRINT © FORTINET
Operation Modes
Transparent Mode Port1 and port3 of a FortiGate in transparent mode are connected to a network. An administrator wants to create 4 broadcast domains. For that purpose, the administrator segmented the network into 4 VLANs: VLAN Name
VLAN tag
FortiGate interfaces
Native VLAN
No tag
port1 port3
VLAN 20
20
port1-VLAN20 port3-VLAN20
VLAN 30
30
port1-VLAN30 port3-VLAN30
VLAN 40
40
port1-VLAN40 port3-VLAN40
The Win-Student server is connected to the native VLAN in port 3. The following diagram summarizes this network topology:
1. First, check that Firefox is not configured to use an explicit web proxy. 2. Click Open menu. Then click Options:
FortiGate III Student Guide
56
DO NOT REPRINT © FORTINET
Operation Modes
Go to Advanced -> Network and click Settings:
Check that No proxy is selected and click OK. 3. Log on to the Student FortiGate’s GUI using the account admin with no password: http://10.0.1.254 4. Upload the Student configuration file for this lab: Resources\FortiGate III\Operation-Modes\Student\student-operationmodes-transparent.conf The Student FortiGate will reboot. Wait a few minutes until it is back up. This changes the FortiGate to transparent mode and adds all the VLAN sub-interfaces. 5. Connect to the Student FortiGate using SSH and start this sniffer: diagnose sniffer packet any "arp and host 10.0.1.15" 4 FortiGate III Student Guide
57
DO NOT REPRINT © FORTINET
Operation Modes
6. From Win-Student command prompt, do a ping to 10.0.1.15: > ping 10.0.1.15 This IP address is not active, so you will not receive any echo reply. However, the ping triggers ARP traffic that can be captured by the previous sniffer. 7. The output of the sniffer will be similar to this:
So, broadcast traffic is being forwarded to all the VLAN sub-interfaces. Each VLAN is not a different broadcast domain, as the administrator wants. Why is this happening? What configuration change must be done in the FortiGate to actually make each VLAN a different broadcast domain? 8. From the FortiGate CLI, execute these configuration changes: # config system interface edit port1-VLAN20 set forward-domain 20 next edit port1-VLAN30 set forward-domain 30 next edit port1-VLAN40 FortiGate III Student Guide
58
DO NOT REPRINT © FORTINET
Operation Modes
set forward-domain 40 next edit port3-VLAN20 set forward-domain 20 next edit port3-VLAN30 set forward-domain 30 next edit port3-VLAN40 set forward-domain 40 next end 9. Execute the sniffer and ping one more time. Now you will see that the ARP packets are confined only to the native VLAN.
FortiGate III Student Guide
59
DO NOT REPRINT © FORTINET
Operation Modes
NAT/Route Mode 1. Log on to the Student FortiGate’s GUI using the account admin with no password: http://10.0.1.254 2. Upload the Student configuration file for this lab: Resources\FortiGate III\Operation-Modes\Student\student-operationmodes-NAT.conf The Student FortiGate will reboot. Wait a few minutes until it is back up. 3. Log on to the Remote FortiGate’s GUI using the account admin with no password: http://10.200.3.1 4. Upload the Remote configuration file for this lab: Resources\FortiGate III\Operation-Modes\Remote\remote-operationmodes-NAT.conf The Remote FortiGate will reboot. Wait a few minutes until it is back up. 5. Check the IPsec VPN configuration in both FortiGate units. Go to VPN -> IPsec -> Tunnels. Check also the firewall policy and the routing table in both devices. Go to Policy & Objects -> Policy -> IPv4, then check Router -> Monitor -> Routing Monitor. You will notice that there is an IPsec VPN created between both units to encrypt the traffic between the subnets 10.0.1.0/24 and 10.0.2.0/24. You will also see a route in the Student FortiGate to the subnet 10.0.2.0/24 using the IPsec tunnel. 6. Execute a continuous ping from the Win-Student command prompt to Win-Remote: > ping -t 10.0.2.10 You will receive the echo reply from Win-Remote as an indication that the tunnel is operating normally. 7. Without stopping the ping, access the Remote FortiGate and go to System -> Network -> Interfaces. Click the plus icon besides port4 to expand it, and edit the interface ToStudent:
FortiGate III Student Guide
60
DO NOT REPRINT © FORTINET
Operation Modes
Change the Administrative Status of this interface to Down. Click OK. 8. Wait a few seconds and then check the status of the VPN in the Student FortiGate. Go to VPN -> Monitor -> IPsec Monitor. As the remote virtual IPsec interface is administratively down, the VPN is down. Check now the routing table. As the VPN is down, the route to 10.0.2.0/24 was removed. Check also the ping running in Win-Student. It is failing. 9. Proceed to bring back up the remote IPsec interface. Access the Remote FortiGate, go to System -> Network -> Interface and edit the ToStudent interface. Change the Administrative Status to Up and click OK. 10. Go back to the Student FortiGate and check the status of the VPN. Go to VPN -> Monitor -> IPsec Monitor. If the VPN is still down, right click it and select Bring Up. The tunnel will come up. 11. Check the routing table. Go to Router -> Monitor -> Routing Monitor. You will notice that the route to the subnet 10.0.2.0/24 is back to the routing table. 12. Check one more time the ping running in Win-Student. It is not working yet. 13. Sniffer this traffic. Connect to the Student FortiGate's CLI and execute this command: # diagnose sniffer packet any "icmp and host 10.0.2.10" 4 Why is the ping not working if the VPN is up and the route is back? Why is the FortiGate still routing the ping traffic through out port1 (and not through ToRemote)? What can be done to prevent this problem?
FortiGate III Student Guide
61
DO NOT REPRINT © FORTINET
Operation Modes
Break and Fix: NAT/Route Mode 1. Log on to the Remote FortiGate’s GUI using the account admin with no password: http://10.200.3.1 2. Upload the Remote configuration file for this lab: Resources\FortiGate III\Operation-Modes\Remote\remote-operationmodes-NAT.conf The Remote FortiGate will reboot. Wait a few minutes until it is back up. 3. Log on to the Student FortiGate’s GUI using the account admin with no password: http://10.0.1.254 4. Upload the Student configuration file for this lab: Resources\FortiGate III\Operation-Modes\Student\student-operationmodes-NAT.conf The Student FortiGate will reboot. Wait a few minutes until it is back up. . The administrator is reporting two problems: 1. You cannot ping 10.200.4.1 from Win-Student 2. The Student FortiGate configuration includes two default routes, one using port1, and the other one using port2. However, only one of them is active in the routing table Can you fix these two problems?
Tips for Troubleshooting
Sniffer the ping to 10.200.4.1
Use the debug flow while running the ping to 10.200.4.1
Use these commands to check the routing table: # get router info routing-table database # get router info routing-table all
Check the status of the link health monitors (if any) under System -> Monitor -> Link Monitor
FortiGate III Student Guide
62
DO NOT REPRINT © FORTINET
External BGP
External BGP During this lab you will troubleshoot some BGP issues between two FortiGate devices.
Objectives
Monitor and check the status of a BGP communication
Troubleshoot some common external BGP issues
Time to Complete Estimated: 30 minutes
FortiGate III Student Guide
63
DO NOT REPRINT © FORTINET
External BGP
Break and Fix: BGP 1. Log on to the Remote FortiGate’s GUI using the account admin with no password: http://10.200.3.1 2. Upload the Remote configuration file for this lab: Resources\FortiGate III\BGP\Remote\remote-BGP.conf The Remote FortiGate will reboot. Wait a few minutes until it is back up. 3. Log on to the Student FortiGate’s GUI using the account admin with no password: http://10.0.1.254 4. Upload the Student configuration file for this lab: Resources\FortiGate III\BGP\Student\student-BGP.conf The Student FortiGate will reboot. Wait a few minutes until it is back up. An administrator has configured BGP between Student and Remote. The Student FortiGate belongs to the autonomous system 65500 and the Remote FortiGate belongs to the autonomous system 65001. However, the BGP peering is currently down. The objective is to bring up the BGP connection between both units. Also, each FortiGate must advertise all its locally connected subnets. Try not to compare both BGP configurations to find mismatches. You should troubleshoot the problem using the BGP debug commands learned during this lesson. Explain each problem supporting your arguments with the output of sniffers and BGP debug commands.
Tips for Troubleshooting
Use these BGP debug commands and sniffer: # get router info routing-table all # get router info bgp summary # get router info bgp network # get router info bgp neighbors # get router info bgp neighbors advertise # diagnose sniffer packet any “port 179” 4
Use the BGP real time debug: # diagnose debug enable
FortiGate III Student Guide
64
DO NOT REPRINT © FORTINET
External BGP
# diagnose ip router bgp all enable # diagnose ip router bgp level info
Use this command to restart the BGP connection any time: # execute router clear bgp all Stop and Think After fixing the BGP connectivity, you might notice that you cannot reach Win-Remote from Win-Student yet, even when both FortiGate routing tables are ok. You do not need to fix this problem during this lab, but can you find out what is causing this issue?
FortiGate III Student Guide
65
DO NOT REPRINT © FORTINET
OSPF
OSPF During this lab you will troubleshoot some OSPF over IPsec issues between two FortiGate devices.
Objectives
Establish OSPF adjacency between FortiGate devices
Use debug commands to troubleshoot some OSPF problems
Monitor the status of a OSPF network
Time to Complete Estimated: 40 minutes
FortiGate III Student Guide
66
DO NOT REPRINT © FORTINET
OSPF
Break and Fix: OSPF 1. Log on to the Remote FortiGate’s GUI using the account admin with no password: http://10.200.3.1 2. Upload the Remote configuration file for this lab: Resources\FortiGate III\OSPF\Remote\remote-OSPF.conf The Remote FortiGate will reboot. Wait a few minutes until it is back up. 3. Log on to the Student FortiGate’s GUI using the account admin with no password: http://10.0.1.254 4. Upload the Student configuration file for this lab: Resources\FortiGate III\OSPF\Student\student-OSPF.conf The Student FortiGate will reboot. Wait a few minutes until it is back up. An administrator has configured an IPsec tunnel between the Student FortiGate and the Remote FortiGate. OSPF has been configured to run over the tunnel, so that each FortiGate can advertise its networks to its remote peer. The tunnel is currently up, however the OSPF adjacency is down. The objective is to have the OSPF routes correctly learned by both FortiGate units. Also, the IPsec VPN must remain stable. Try not to compare both OSPF configurations to find mismatches. You should troubleshoot the problem using the OSPF debug commands learned in this lesson. Explain each problem supporting your arguments with the output of the debug commands.
Tips for Troubleshooting
Check the routing table and OSPF neighbor status: # get router info routing-table all # get router info ospf status # get router info ospf neighbor Is the neighbor adjacency established? Are OSPF routes present?
Run the real time debug: # diagnose ip router ospf all enable # diagnose ip router ospf level info # diagnose debug enable
FortiGate III Student Guide
67
DO NOT REPRINT © FORTINET
OSPF
Once the OSPF issues are resolved, go to the VPN event logs. Is the IPsec VPN stable? Watch the log messages for a few minutes Compare the Student routing table when the tunnel is down with the table when it is up. What is causing the tunnel to bounce?
FortiGate III Student Guide
68
DO NOT REPRINT © FORTINET
High Availability
High Availability During this lab you will troubleshoot some high availability problems between two FortiGate devices.
Objectives
Monitor a HA cluster
Check the status of the HA configuration and session synchronization
Troubleshoot some common HA problems
Time to Complete Estimated: 30 minutes
FortiGate III Student Guide
69
DO NOT REPRINT © FORTINET
High Availability
Break and Fix: High Availability
FortiGate
REMOTE port3
port1 LINUX 10.200.1.254 eth1
port2
port2
port3 10.0.1.254/24 WIN-STUDENT 10.0.1.10
port1 10.200.1.1/24
STUDENT FortiGate
LAN3 0.0.0.0
eth0 LAN0 0.0.0.0
1. Log on to the Remote FortiGate’s GUI using the account admin with no password: http://10.200.3.1 2. Upload the Remote configuration file for this lab: Resources\FortiGate III\High-Availability\Remote\remote-ha.conf The Remote FortiGate will reboot. Wait a few minutes until it is back up. 3. Log on to the Student FortiGate’s GUI using the account admin with no password: http://10.0.1.254 4. Upload the Student configuration file for this lab: Resources\FortiGate III\High-Availability\Student\student-ha.conf The Student FortiGate will reboot. Wait a few minutes until it is back up. After loading both configurations, the cluster is not forming. The Remote unit cannot join the HA cluster. Use the debug commands learned in this lesson to troubleshoot the problem.
FortiGate III Student Guide
70
DO NOT REPRINT © FORTINET
High Availability
Tips for Troubleshooting
Run the HA real time debug in the CLI of both units: # diagnose debug application hatalk 255 # diagnose debug application hasync 255 # diagnose debug enable
Use these additional HA debug commands: # diagnose sys ha status # diagnose sys ha showcsum
For easy access to each unit while the cluster is down, each FortiGate starts with different IP addresses in their port3: Student: 10.0.1.254 Remote: 10.0.1.253 So, while Remote cannot join the cluster, you can connect to its port3 IP address via SSH and run the debug commands Stop and Think After the Remote FortiGate joins the cluster, you will notice that you cannot access the Remote FortiGate using the IP address 10.0.1.253 anymore. Can you explain why?
FortiGate III Student Guide
71
DO NOT REPRINT © FORTINET
Appendix A: Additional Resources
Appendix A: Additional Resources Training Services
http://training.fortinet.com
Technical Documentation
http://help.fortinet.com
Knowledge Base
http://kb.fortinet.com
Forums
https://forum.fortinet.com/
Customer Service & Support
https://support.fortinet.com
FortiGuard Threat Research & Response
http://www.fortiguard.com
FortiGate III Student Guide
72
DO NOT REPRINT © FORTINET
Appendix B: Presentation Slides
Appendix B: Presentation Slides
FortiGate III Student Guide
73
DO NOT REPRINT © FORTINET
Troubleshooting Concepts
This lesson is about troubleshooting concepts.
FortiGate III Student Guide
74
DO NOT REPRINT © FORTINET
Troubleshooting Concepts
In this lesson, we will review troubleshooting strategies. We will also introduce some of the troubleshooting tools available in the FortiGate GUI and CLI.
FortiGate III Student Guide
75
DO NOT REPRINT © FORTINET
Troubleshooting Concepts
Let’s being by reviewing some troubleshooting concepts and strategies.
FortiGate III Student Guide
76
DO NOT REPRINT © FORTINET
Troubleshooting Concepts
Good administrators know their network well before any problem happens. That includes an understanding of the normal behavior related with traffic volume, network applications, traffic flows and devices' CPU and memory utilization. So, when a problem happens, good administrators identify quickly what is behaving abnormally. This information speeds up the troubleshooting process and helps to isolate the cause of the problem. Many tools can be used to gather statistics and information while the network is operating normally: SNMP, logging, sFlow, and the monitors located in the FortiGate GUI.
FortiGate III Student Guide
77
DO NOT REPRINT © FORTINET
Troubleshooting Concepts
It is also important to keep the network documentation up-to-date. Network diagrams should include physical connections, interface names and subnets. Good network documentation also includes change control records to track any change in the network: Who did the change? When was done? What was changed?
FortiGate III Student Guide
78
DO NOT REPRINT © FORTINET
Troubleshooting Concepts
If a problem happens, the first step is to define it well. For example, if the problem definition is “web filtering is not working”, the scope of the problem is too imprecise. Too many things could cause this. This makes troubleshooting slow. So, we must ask questions to understand the details: Is the problem happening with one web site? Is it happening with all users? Is it happening randomly? How can you reproduce the problem? After answering the right questions, we can define the problem with details. For example: “the web filtering is not blocking the web site X for the user Y”. This provides a better place to start the troubleshooting.
FortiGate III Student Guide
79
DO NOT REPRINT © FORTINET
Troubleshooting Concepts
A general approach for troubleshooting network issues is to follow the TCP/IP model and work the problem either from the highest layer to the bottom or from the lowest layer to the top. In the first method you check the physical layer first. If a layer operation is ok, you move to the upper layer, until you find the layer where the problem is happening. In the second method you check the application layer first, if a layer is not working properly you move to the layer below to rule out issues in the lower layers.
FortiGate III Student Guide
80
DO NOT REPRINT © FORTINET
Troubleshooting Concepts
During the second part of this lesson, we will review some of the troubleshooting tools available in the FortiGate GUI.
FortiGate III Student Guide
81
DO NOT REPRINT © FORTINET
Troubleshooting Concepts
The dashboard is the FortiGate GUI welcome screen. Some of its widgets contain information useful for troubleshooting, such as the system resources and the alert message console widgets.
FortiGate III Student Guide
82
DO NOT REPRINT © FORTINET
Troubleshooting Concepts
Remember that the dashboard is customizable. Widgets can be added, removed and customized.
FortiGate III Student Guide
83
DO NOT REPRINT © FORTINET
Troubleshooting Concepts
In addition to the dashboard’s widgets, the GUI includes some monitor screens for specific features. For example, the IPsec monitor displays the status of each IPsec tunnel. The firewall monitor shows the list of authenticated users. Another example is the routing monitor, which lists the routes that have been loaded (active) the routing table.
FortiGate III Student Guide
84
DO NOT REPRINT © FORTINET
Troubleshooting Concepts
Another important tool for troubleshooting is the FortiGate logs. The log viewer includes a filter setting that is used to display only the logs entries related with one specific user name, IP address, URL or event type.
FortiGate III Student Guide
85
DO NOT REPRINT © FORTINET
Troubleshooting Concepts
This table illustrates the expected log behavior depending on the different logging settings. The first column shows the possible values for the log setting in the firewall policies: no log, log security events, or log all sessions. The second column indicates if the AV, web filtering or antispam profile log setting is enabled or disabled. Remember, DLP and IPS profiles always generate logs in the security log section. The last column shows the behavior. If you enable any profiles on your policy and logging is not enabled, you will not get logs of any kind—even when profile is blocking traffic. So if you apply a security profile, it’s important to consider the logging settings.
FortiGate III Student Guide
86
DO NOT REPRINT © FORTINET
Troubleshooting Concepts
From the information stored in the logs, a FortiGate device with hard disk can generate reports, either manually or on an automatic schedule.
FortiGate III Student Guide
87
DO NOT REPRINT © FORTINET
Troubleshooting Concepts
Reports in FortiGate devices are fully customizable.
FortiGate III Student Guide
88
DO NOT REPRINT © FORTINET
Troubleshooting Concepts
Let's now introduce some CLI troubleshooting tools.
FortiGate III Student Guide
89
DO NOT REPRINT © FORTINET
Troubleshooting Concepts
The real time debug commands generate information in real time about what a specific FortiGate process or feature is doing. The debug level is a bitmask value that specifies which types of messages are displayed. This depends on each process. For all the cases, although, a debug level of ‘0’ means no output (disabled), and a debug level of '-1' means enabling all possible message types.
FortiGate III Student Guide
90
DO NOT REPRINT © FORTINET
Troubleshooting Concepts
For example, this slide shows the two commands for enabling all the IPsec real time debug output. You can also enable the option to prepend the system time to each debug line. It is important to disable any real time debug after using it. They consume FortiGate resources and some could be CPU intensive.
FortiGate III Student Guide
91
DO NOT REPRINT © FORTINET
Troubleshooting Concepts
On the other hand, the application layer test commands do not display information in real time, but statistics and configuration information about a feature or process. Some of these commands can also be used to restart a process or execute a change in its operation.
FortiGate III Student Guide
92
DO NOT REPRINT © FORTINET
Troubleshooting Concepts
Some CLI debug commands generate a lot of output. If we know that the required information is contained in one specific line of the output, and if we know a keyword to find that line, we can use the GREP utility. This utility displays only the lines from the output that match a text string. For example, in this slide we are using the GREP utility to find the IP address 10.0.0.7 in the ARP table. We use the debug command ‘diagnose ip arp list’ to get the ARP table, and then we append the GREP utility to display only the information for one IP address. In this way, the output is only one line (with exactly the information we need), instead of a long list of entries.
FortiGate III Student Guide
93
DO NOT REPRINT © FORTINET
Troubleshooting Concepts
When displaying the FortiGate configuration via CLI, you can use the GREP utility with the option –f. It will display only the configuration sections (or tables) where the text string matches at least one value. This is useful, for example, to find all the references in the configuration to a specific object. In this slide, we are using the –f option to find all the references to the wan1 interface. The output shows only the two tables where wan1 is referenced: the definition of the interface itself, and a firewall policy where wan1 is the destination interface.
FortiGate III Student Guide
94
DO NOT REPRINT © FORTINET
Troubleshooting Concepts
If you need to expand your FortiGate skills, or if you need more information about troubleshooting a FortiGate issue, these are some of the resources to get more information and help.
FortiGate III Student Guide
95
DO NOT REPRINT © FORTINET
Troubleshooting Concepts
To review, these are the topics that we just talked about.
FortiGate III Student Guide
96
DO NOT REPRINT © FORTINET
System Resources
In this lesson, we will show you how to troubleshoot system resources on FortiGate.
FortiGate III Student Guide
97
DO NOT REPRINT © FORTINET
System Resources
You will learn some debug commands for troubleshooting FortiGate system problems, such as high CPU or memory usage. The lesson also covers new firmware installations from the BIOS, memory optimization and crashlog diagnostics.
FortiGate III Student Guide
98
DO NOT REPRINT © FORTINET
System Resources
Let’s see first some general system troubleshooting commands.
FortiGate III Student Guide
99
DO NOT REPRINT © FORTINET
System Resources
This is usually one of the first debug commands when troubleshooting. The output shows the firmware version, FortiGuard database versions, license status, operation mode, number of VDOMs and system time.
FortiGate III Student Guide
100
DO NOT REPRINT © FORTINET
System Resources
The command 'get system performance status' shows the overall memory and CPU utilization. It also shows session creation rate, number of viruses caught and number of attacks blocked by the IPS. The last line displays the system uptime. This output gives a quick overlook at how much traffic the unit is handling.
FortiGate III Student Guide
101
DO NOT REPRINT © FORTINET
System Resources
During this part of the lesson you will learn how to check the use of he FortiGate memory and CPU.
FortiGate III Student Guide
102
DO NOT REPRINT © FORTINET
System Resources
To understand how a FortiGate uses its memory, we need to understand the architecture of FortiOS. The hearth of FortiOS is its kernel. Here, the unit takes some of the most important and basic decisions, such as how to route a packet, or when to offload a session to a NPU processor. FortiOS runs over hardware. The device drivers bridge the kernel with the hardware. Above the kernel, there is the user space. Several application processes or daemons run at this level. Above all that, there is the configuration layer, which is basically composed of two modules: the command line interface and the graphical user interface.
FortiGate III Student Guide
103
DO NOT REPRINT © FORTINET
System Resources
How the memory is segmented depends on the FortiGate model. There are two cases: first, models running 32-bit FortiOS with more than 1 GB of memory; second, models running 64-bit FortiOS and models running 32-bit FortiOS with less than 1 GB of memory. Let’s see the first case. When the FortiOS is 32 bits and the system memory is more than 1 GB, the kernel cannot access directly the whole memory space. So, memory paging is used to reach the portion of the memory that cannot be accessed directly. The part of the memory that the kernel can access directly is called low memory. The part of the memory that is accessed using memory paging is called high memory. The command 'diagnose hardware sysinfo memory' displays: - The total amount of low memory (LowTotal) - The amount of free low memory (LowFree) - The total amount of high memory (HighTotal) - The amount of free high memory (HighFree) - The total amount of system memory (MemTotal = LowTotal + HighTotal) - The total amount of free memory (MemFree = LowFree + HighFree)
FortiGate III Student Guide
104
DO NOT REPRINT © FORTINET
System Resources
For models running 32-bit FortiOS with less than 1 GB, and for models running 64-bit FortiOS, the kernel doesn't need to use memory paging to access the whole memory space. In those cases the command 'diagnose hardware sysinfo memory' shows a size of 0 kB for the total high memory and free high memory. All the memory space is considered low memory.
FortiGate III Student Guide
105
DO NOT REPRINT © FORTINET
System Resources
FortiGate allocates memory for five main purposes: - Kernel memory slabs - System I/O cache - Buffers - Shared memory - Process memory We will see each of these in the next slides.
FortiGate III Student Guide
106
DO NOT REPRINT © FORTINET
System Resources
The kernel memory slabs are collections of objects with a common purpose. They are used by the kernel to store information in the low memory. This slide shows example of some slabs. There are slabs for storing information about the TCP sessions. The entries in the route cache are also stored in low-memory slabs.
FortiGate III Student Guide
107
DO NOT REPRINT © FORTINET
System Resources
(slide contains animation) To check how much memory is being allocated to kernel slabs, we use the command 'diagnose hardware sysinfo slab'. The first column shows the slab name. The second column the total amount of active objects, (click) then the amount of available objects, (click) and the size of each object. The total amount of memory allocated to each slab type can be calculated by multiplying the number of available objects by their size.
FortiGate III Student Guide
108
DO NOT REPRINT © FORTINET
System Resources
The system I/O cache is used to speed up the access to information stored in the hard and flash disk memories. Some processes, such as logging, WAN optimization, and explicit proxy, store information in the hard disk, so they get the performance boost provided by this memory allocation. An I/O cache page is labeled as active when it has been recently accessed. It goes to inactive state after not been used for some time. An inactive page can be reclaimed by the kernel if needed.
FortiGate III Student Guide
109
DO NOT REPRINT © FORTINET
System Resources
(slide contains animation) The command 'diagnose hardware sysinfo memory' displays the total amount of memory allocated for I/O cache. (click) It also lists the amount of memory allocated for buffers.
FortiGate III Student Guide
110
DO NOT REPRINT © FORTINET
System Resources
As we explained, above the kernel layer there are multiple application processes or daemons running. The operating system allocates separated blocks of memory for each process. One process can access the memory that was allocated to it, but it cannot access any memory that was allocated to a different process. So, a process cannot share information with another process by reading or writing data into the memory allocated to that other process. For that purpose, the operating system dynamically allocates shared memory (SHM). The SHM can be accessed by multiple processes, allowing the sharing the information among them.
FortiGate III Student Guide
111
DO NOT REPRINT © FORTINET
System Resources
We can check how much memory space is being used by each process. The command 'diagnose sys top' shows that information in the last column. The command also displays, for each process: its ID number, its state, and the amount of CPU using. You can specify the refresh frequency and the number of lines to display. While the command is running, you can press to sort the processes by CPU usage, of to sort them by memory usage. To stop the command, use .
FortiGate III Student Guide
112
DO NOT REPRINT © FORTINET
System Resources
Another useful command for displaying information about process memory usage is 'diagnose sys topsummary'. The –h option displays the different options available for this command.
FortiGate III Student Guide
113
DO NOT REPRINT © FORTINET
System Resources
(slide contains animation) For example we can use the option –s mem to sort the processes by memory usage; the option –i 60 to refresh it every 60 seconds; and the option –n 10 to display the top 10 processes. One advantage of this command over the previous one ('diagnose sys top') is that it shows the total amount of memory allocated by all forked or child processes, including shared memory. In the case of 'diagnose sys top', each child process is displayed separated. Here, we have an aggregated view of all of them. (click) The output also shows the amount of file descriptors allocated. If the amount of any of the descriptors keeps constantly increasing, it might indicate that there is a memory leak problem. (click) If a process has forked, the number of child processes is displayed after its name.
FortiGate III Student Guide
114
DO NOT REPRINT © FORTINET
System Resources
This table shows some of the most common processes.
FortiGate III Student Guide
115
DO NOT REPRINT © FORTINET
System Resources
This other table shows more about the most common processes.
FortiGate III Student Guide
116
DO NOT REPRINT © FORTINET
System Resources
The command 'diagnose sys top' shows the state of each process. A process can be in one of four states: Sleeping (S), running (R), do not disturb (D) or zombie (Z). The S and R states are normal. It is also normal if a process goes briefly to D state. The Z state is not normal. Also, it is not normal if a process stays in D state for a long time. These usually indicate that the process is not working properly.
FortiGate III Student Guide
117
DO NOT REPRINT © FORTINET
System Resources
During this part of the lesson you will learn about conserve mode.
FortiGate III Student Guide
118
DO NOT REPRINT © FORTINET
System Resources
There are 2 different types of conserve modes: - Kernel conserve model is triggered when the amount of free low memory is running low - Proxy conserve mode (also called system conserve mode) is triggered when the amount of free overall memory is running low. The command 'diagnose hardware sysinfo shm' can be used to determine the conserve mode status. If the field conservemode is 1, the unit is in proxy conserve mode. If it is 2, the unit is in kernel conserve mode.
FortiGate III Student Guide
119
DO NOT REPRINT © FORTINET
System Resources
Two margins or thresholds determine when the FortiGate enters and exists the kernel conserve mode. The margins depend on the total amount of low memory. When a FortiGate is in kernel conserve mode, any proxy inspection is bypassed and administrators cannot change the unit configuration.
FortiGate III Student Guide
120
DO NOT REPRINT © FORTINET
System Resources
These are the entries generated in the crashlog, event logs and alert message console when a FortiGate enters to and leaves the kernel conserve mode.
FortiGate III Student Guide
121
DO NOT REPRINT © FORTINET
System Resources
Similarly to kernel conserve mode, two margins or thresholds determine when the FortiGate enters and exists the proxy conserve mode. The margins depend on the total amount of overall memory.
FortiGate III Student Guide
122
DO NOT REPRINT © FORTINET
System Resources
These are the entries generated in the crashlog and event logs when a FortiGate enters to and leaves the proxy conserve mode.
FortiGate III Student Guide
123
DO NOT REPRINT © FORTINET
System Resources
av-fail-open is the CLI setting that controls FortiGate’s behavior while it is in proxy conserve mode.
FortiGate III Student Guide
124
DO NOT REPRINT © FORTINET
System Resources
An option related to fail-open, is av-failopen-session. This is a setting that kicks in not during a high memory situation, but when a proxy on the FortiGate runs out of available sessions to process the traffic. If av-failopen-session is enabled, then FortiGate will act according to the av-failopen setting. Otherwise, by default, it will block new sessions until proxy connections become available.
FortiGate III Student Guide
125
DO NOT REPRINT © FORTINET
System Resources
Additionally, FortiGate has one more mechanism to free memory when there is not much available. If the kernel cannot allocate more memory pages, it deletes the oldest sessions. The command diagnose sys session stat shows the numbers of sessions that has been deleted by the kernel due to this mechanism.
FortiGate III Student Guide
126
DO NOT REPRINT © FORTINET
System Resources
During this part of the lesson you will learn how to optimize the memory usage.
FortiGate III Student Guide
127
DO NOT REPRINT © FORTINET
System Resources
Many FortiGate processes, such as DLP or AV scanning, are memory intensive. So, memory optimization is important, especially in small units, to guarantee that they will stay away from conserve mode. This slide shows some recommendations for optimizing the memory usage. These tips might increase significantly the available memory in a device that is frequently going to conserve mode. The first and most logical step is to disable the features that are not required. For example, if the network already has a FortiMail doing antispam, an administrator does not need to do antispam in the FortiGate. Also, usually not all the IPS signatures are required. Another important recommendation is to reduce the maximum file size to inspect, which is set to 10MB by default. This value can be reduced to 2 or 3 MB without reducing significantly the virus caching rate.
FortiGate III Student Guide
128
DO NOT REPRINT © FORTINET
System Resources
Additionally, you can reduce the amount of allocated memory to some caches, such as the ones for FortiGuard and DNS.
FortiGate III Student Guide
129
DO NOT REPRINT © FORTINET
System Resources
The FortiGate session table, especially in networks with high traffic, can consume an important portion of the memory. By default, a session without traffic remains in the table for up to 1 hour. Although this high time to live (TTL) might be required by a few applications, in most of the networks, it can be substantially reduced to, for example, 5 minutes. So, FortiGate will age out idle sessions much quicker, increasing the amount of available memory. There are four places in the FortiGate configuration where the session TTL can be changed. Two of them are: - Globally for all the traffic - Per IP protocol and port number
FortiGate III Student Guide
130
DO NOT REPRINT © FORTINET
System Resources
The other two places where the session TTL can be changed are: - Per firewall policy - Per application control If there is one specific application that requires a high session TTL, you can, for example, reduce the TTL globally to 5 minutes, and set it to 1 hour only for the specific application port number, or firewall policy.
FortiGate III Student Guide
131
DO NOT REPRINT © FORTINET
System Resources
In the case of TCP, there are other session timers. In most of the cases, they can also be reduced from their default values without causing problems with the applications. The slide shows some recommended values (equal to or below the default ones) to optimize the memory usage. We will see what each of those timers are in the next slide.
FortiGate III Student Guide
132
DO NOT REPRINT © FORTINET
System Resources
The tcp-halfopen-timer controls for how long, after a SYN packet, a session without SYN/ACK remains in the table. The tcp-halfclose-timer controls for how long, after a FIN packet, a session without FIN/ACK remains in the table. The tcp-timewait-timer controls for how long, after a FIN/ACK packet, a session remains in the table. A closed session remains in the session table for a few seconds more to allow any out-of-sequence packet.
FortiGate III Student Guide
133
DO NOT REPRINT © FORTINET
System Resources
To finish this lesson, we will talk about hardware and firmware troubleshooting.
FortiGate III Student Guide
134
DO NOT REPRINT © FORTINET
System Resources
Let’s talk about the crashlog. Each time an application crashes, or closes, an entry is generated in the crashlog. For the case of crashes, the entry contains information about the name of the application, the time it crashed and the termination signal. This slide shows a sample of a crash in the crashlog. In this case the application that failed was the sslvpnd process, which manages SSL VPN connections. The termination signal is 11, which is segmentation fault.
FortiGate III Student Guide
135
DO NOT REPRINT © FORTINET
System Resources
This table contains the most common termination signal numbers. Any administrator can manually kill a process by using the command 'diagnose sys kill', followed by the process ID and the termination signal number. The command 'diagnose sys top' lists the process ID numbers. Manually killing a process is not usually required under normal circumstances. Be careful although if, for some reason, you have to do it. Improperly killing a process might make a FortiGate system unstable.
FortiGate III Student Guide
136
DO NOT REPRINT © FORTINET
System Resources
So, how de we know if the crashlog is normal or not?. In most of the cases, entries in the crashlog are normal. A crashlog entry can be considered suspicious if it happens at the same time than a FortiGate feature that failed or an abnormal FortiGate behaviour. For example, a crashlog entry generated when the unit unexpectedly reboots might provide information about the cause. A crash in the sslvpnd process when all SSL VPN users got disconnected is also relevant. The crashlog includes information that can be used by Fortinet development to determine which code triggered the problem and the details about the crash.
FortiGate III Student Guide
137
DO NOT REPRINT © FORTINET
System Resources
If a FortiGate that is rebooting unexpectedly, this is what an administrator can do: First, check the logs and the crashlog. Second, a crashdump message is usually generated through the console port when the unit crashes. This crashdump messages can provide useful information to the Fortinet developers. To capture any crashdump, keep a laptop connected to the console port, run the commands in this slide, and wait until another crash happens.
FortiGate III Student Guide
138
DO NOT REPRINT © FORTINET
System Resources
We say that a FortiGate freezes when stops handling traffic and there is no way to connect to it (there is even not access to the unit's console port). Only power cycling fixes the problem. For those cases, you could similarly capture any crashdump in the console port. Additionally, and in the case of models with more than one CPU, you can enable the nmi-watchdog feature, which crashes the system (and forces the crashdump) when no more new daemons have been scheduled in the last 10 minutes - This is an indication that the unit is not operating normally and might have frozen. Some FortiGate models also have an external NMI button. If the unit is frozen and no crashdump has been generated, you can press it to force a crash and get a crashdump.
FortiGate III Student Guide
139
DO NOT REPRINT © FORTINET
System Resources
The FortiGate BIOS menu allows administrators to do some operations over the flash memory and the firmware images. To access the BIOS menu you must reboot the unit while connected to the console port. The console port will show the booting process and, at one point, it will show the message: “Press any key to display configuration menu” If you press any key while this prompt is displayed, the booting process is interrupted and the BIOS menu is displayed.
FortiGate III Student Guide
140
DO NOT REPRINT © FORTINET
System Resources
The options available in the BIOS menu vary depending of the BIOS version and hardware model. However, in most of the cases you will see an output similar to the one in this slide.
FortiGate III Student Guide
141
DO NOT REPRINT © FORTINET
System Resources
By pressing F we can format the flash memory. This might be required if the firmware got corrupted or if the administrator wants to do a clean installation of a new firmware. Keep in mind, although, that formatting the flash deletes any information stored, such as firmware images, configuration and digital certificates.
FortiGate III Student Guide
142
DO NOT REPRINT © FORTINET
System Resources
Fallow these steps to install a firmware image from the BIOS.
FortiGate III Student Guide
143
DO NOT REPRINT © FORTINET
System Resources
After that, go to the BIOS menu and select the option G. The BIOS will ask for: - The IP address of the TFTP server (configure the TFTP server with this IP address) - The FortiGate IP address (it must be in the same class-C subnet than the TFTP server) - The name of the firmware image file stored in the TFTP server If everything is ok, you should see a series of pound signs, indicating that the unit is downloading the image. The BIOS will then verify the integrity of the file and, in some models, will give you these three options: - Save it as the default firmware - Save it as the backup firmware - Run the image without saving it
FortiGate III Student Guide
144
DO NOT REPRINT © FORTINET
System Resources
There is a hardware test images (HQIP) for each model. You can install it by using the BIOS menu and a TFTP server. You can run it without saving it to the flash. The HQIP runs some hardware diagnostics to test the NIC interfaces, hard disk, flash disk and other hardware components.
FortiGate III Student Guide
145
DO NOT REPRINT © FORTINET
System Resources
These are the topics covered in this lesson.
FortiGate III Student Guide
146
DO NOT REPRINT © FORTINET
Network
This lesson is about network troubleshooting.
FortiGate III Student Guide
147
DO NOT REPRINT © FORTINET
Network
You will learn to use some general network troubleshooting tools, the built-in sniffer and the debug flow. The lesson also shows how to display and analyze the information in the FortiGate session table.
FortiGate III Student Guide
148
DO NOT REPRINT © FORTINET
Network
The first part of this lesson introduces some general network troubleshooting tools.
FortiGate III Student Guide
149
DO NOT REPRINT © FORTINET
Network
This is one of the most useful commands for troubleshooting layer-1 problems. It displays the link status, negotiated speed and negotiated duplex mode. It also shows counters related with bytes, packets and errors received and transmitted. The output of this command varies depending on the FortiGate model and NIC driver version.
FortiGate III Student Guide
150
DO NOT REPRINT © FORTINET
Network
This table contains some of the fields in the output of the 'get hardware nic' command. Not all these are displayed for the same NIC. As we explained, the output depends on the model and NIC driver version.
FortiGate III Student Guide
151
DO NOT REPRINT © FORTINET
Network
This other table contains more fields that might be present in the output of the 'get hardware nic' command.
FortiGate III Student Guide
152
DO NOT REPRINT © FORTINET
Network
To check the ARP table, use the command 'get system arp'. It displays the IP addresses, MAC addresses and interfaces where the devices are connected. It also shows the age, which is for how long there has not been traffic. After a predefined time, an entry in the ARP table without traffic is aged out.
FortiGate III Student Guide
153
DO NOT REPRINT © FORTINET
Network
Two of the most common troubleshooting tools are the 'ping' and the 'traceroute'. Remember from our NSE4 training that the options for the ping can be changed with the command 'execute ping-options'. They include the packet size, amount, timeout and the source IP address for the ping.
FortiGate III Student Guide
154
DO NOT REPRINT © FORTINET
Network
The CLI also includes a telnet client, which cannot only be used to connect to a remote telnet device, but could also be used to test the connectivity to an application server. For example, in this slide we are testing the connectivity from the FortiGate to a SMTP server by doing telnet to the server's port 25.
FortiGate III Student Guide
155
DO NOT REPRINT © FORTINET
Network
The command 'get system performance firewall statistics' displays aggregated statistics per network application.
FortiGate III Student Guide
156
DO NOT REPRINT © FORTINET
Network
The command 'get system performance firewall packet-distribution' displays number of bytes and packets per packet size range.
FortiGate III Student Guide
157
DO NOT REPRINT © FORTINET
Network
In the last part of this lesson, we will talk about the FortiGate session table, the built-in sniffer and the debug flow.
FortiGate III Student Guide
158
DO NOT REPRINT © FORTINET
Network
The FortiGate session table, as explained in the NSE4 training, contains detailed information about every IP connection crossing or terminating at the FortiGate. The command 'get system session status' displays the total number of sessions in the active VDOM. The command 'get system session list' provides a brief summary of each session, one session per line, with the most relevant information, such as protocol, and source and destination IP address and port. We can use the GREP utility with this command, for example, to list only the sessions for a specific IP address.
FortiGate III Student Guide
159
DO NOT REPRINT © FORTINET
Network
To display detailed information about the sessions, we use the command 'diagnose sys session list'. It is recommended to set the session filter first, as an unfiltered output displays all the details about all the existing sessions (which could be in order of thousands, or even millions, for high-end devices). You can filter the output, for example, by policy ID, or by source or destination IP address and port.
FortiGate III Student Guide
160
DO NOT REPRINT © FORTINET
Network
Some configuration changes, such as security profile or session helper changes, apply only to new sessions. For those cases, existing sessions keep using the old configuration until they expire or are terminated. This is important when troubleshooting problems. For example, after a change in a security profile, you should clear any related existing session and generate new ones. Use the command 'diagnose sys session clear' to remove all sessions that match the session filter. You must although be careful with this command as it could potentially clear all the existing sessions if no filter has been set. Double check first the session filter with the command 'diagnose sys session filter'.
FortiGate III Student Guide
161
DO NOT REPRINT © FORTINET
Network
(slide contains animation) The slide shows a sample of the information contained in the FortiGate session table. It includes the IP protocol number, (click) the protocol state (we will see what this value is in the next slides), (click) how long for the session to expire (if there is not more traffic), (click) traffic shaping counters, (click) session flags, (click) received and transmitted packet and byte counters, (click) if the unit is doing NAT - this portion shows the type of NAT (source or destination) for each traffic direction, and the NAT IP address (click) the ID of the matching policy, (click) and counters for hardware acceleration.
FortiGate III Student Guide
162
DO NOT REPRINT © FORTINET
Network
The protocol state in the session table is a two-digit number. In the case of TCP, the first number is related with the client-side state, and is different than 0 when the session is being handled by a FortiGate proxy (for example, when the FortiGate is doing proxy-based inspection). The second digit is the server-side state and is used to know the status of the TCP session. This table and the flow graph correlate the second digit value with the different TCP session states. For example when the FortiGate receives the SYN packet, the second digit is 2. It goes to 3 once the SYN/ACK is received. After the three-way handshake, the state value changes to 1. When a session is closed by both sides, the FortiGate keeps it in the session table for a few seconds more, to allow any out-of-order packet that could arrive after the FIN/ACK packet. This is the state value 5.
FortiGate III Student Guide
163
DO NOT REPRINT © FORTINET
Network
For case of UDP, the session state can only have two values: 00 when traffic has been only one way, and 01 once there is traffic two ways. For the case of ICMP the protocol state is always 00.
FortiGate III Student Guide
164
DO NOT REPRINT © FORTINET
Network
This table contains the meaning of the most important session flags. For example the log flag indicates that the session is being logged. The local flag indicates that the session either was originated from the FortiGate or is terminating in the FortiGate.
FortiGate III Student Guide
165
DO NOT REPRINT © FORTINET
Network
Let’s give more details about the dirty and may dirty flags. When the FortiGate receives the first packet for a new session (for example, a SYN packet), the unit evaluates if the traffic should or should not be allowed based on the firewall policies. As long as there are not changes in the firewall policies, this evaluation is done only on the first session packet. If the traffic is allowed by a firewall policy, the unit creates a session and flags it as may_dirty. After that, if there is a change in the firewall policy configuration, all the existing sessions with the may_dirty flag are also flagged as dirty. This indicates to the FortiGate that it needs to reevaluate the next session packet to know if the session must now be blocked. If the session is still allowed, the dirty flag is removed (the may_dirty flag is kept). If the session must be blocked, it is flagged as block and remains in the session table until it expires. Any packet matching a session with the block flag is dropped.
FortiGate III Student Guide
166
DO NOT REPRINT © FORTINET
Network
We talk about the built-in sniffer in NSE4. This slide reviews this tool. There are 6 verbosity levels. The table shows what information is displayed in each level. We usually use level 4 to check how the traffic is flowing and if the FortiGate is not dropping packets. We also use level 3 or 6 to convert the output to PCAP format, which can be later analyzed with a tool such as WireShark.
FortiGate III Student Guide
167
DO NOT REPRINT © FORTINET
Network
(slide contains animation) To sniffer the traffic in all the interfaces, use the any keyword as the interface name. (click) If you do not specify any option for the timestamp, the debug shows the time, in seconds, since it is running. As explained in the previous slide, you can instead prepend the local system time to easily correlate a packet with another recorded event. (click) After stopping the sniffer by pressing Ctrl-C, it is important to check the amount of dropped packets. If there were dropped packets during the sniffer, it means that not all the traffic matching the sniffer filter could be captured. So, you might need to capture the traffic again, next time using a stricter filter, or reducing the traffic to capture.
FortiGate III Student Guide
168
DO NOT REPRINT © FORTINET
Network
Another useful FortiGate troubleshoot tool is the debug flow. We also talk about these commands in NSE4, so we will quickly review them. The debug flow is also called ‘internal sniffer’ as it works similarly to the built-in sniffer. But, in this case, the output shows step-by-step, and with details, what the kernel is doing with each packet.
FortiGate III Student Guide
169
DO NOT REPRINT © FORTINET
Network
(slide contains animation) This is a sample of a debug flow output. Here we have captured the 3 packets of a TCP 3-way handshake. The output for the SYN packet shows when the kernel creates a new session (with its session ID), finds the route to the destination and applies NAT. It also shows the ID of the policy that matches this traffic. (click) The output of the SYN/ACK and ACK packets also shows the session ID and NAT information. This tool is useful for many troubleshooting cases, for example when we need to understand why a packet is taking a specific route, or why a specific NAT IP address is being applied.
FortiGate III Student Guide
170
DO NOT REPRINT © FORTINET
Network
The debug flow is also useful when the FortiGate, for some reason, is dropping packets. In those cases it usually shows an error message explaining why a packet was dropped. These are three common messages that we might see in a debug flow output. The first one means that either there is not firewall policy allowing the traffic or that a disclaimer has not been accepted yet. The second one indicates that the IP address has been quarantined by the DLP inspection. The third one means that the packet was dropped due to a traffic shaper that has exceeded one of its thresholds.
FortiGate III Student Guide
171
DO NOT REPRINT © FORTINET
Network
These are two more common debug flow error messages. The first error indicates that the packet failed the reverse path forwarding check. The second error message is usually because one of these reasons: if the packet is destined to a FortiGate IP address (for example, management traffic), it might be that the service is not enabled, is using a different port, or the source IP address if not included in the trusted list. On the other hand, if the packet is not destined to a FortiGate IP address, but to a device on the other side of the unit, there might be a virtual IP or IP pool that is wrongly using that IP address. So, check your virtual IP or IP pool configuration.
FortiGate III Student Guide
172
DO NOT REPRINT © FORTINET
Network
These are the topics covered in this lesson.
FortiGate III Student Guide
173
DO NOT REPRINT © FORTINET
Firewall Policies
This lesson is about firewall policy troubleshooting.
FortiGate III Student Guide
174
DO NOT REPRINT © FORTINET
Firewall Policies
During this lesson, you will learn about how the FortiGate translates the source port when doing source NAT. The lesson also covers NAT exhaustion, SIP and session helper debug commands. During the last part of the lesson, you will learn some debug commands for troubleshooting traffic shaping.
FortiGate III Student Guide
175
DO NOT REPRINT © FORTINET
Firewall Policies
This part of this lesson is about NAT and NAT exhaustion problems.
FortiGate III Student Guide
176
DO NOT REPRINT © FORTINET
Firewall Policies
This slide shows the different FortiGate NAT methods. The differences between each of them are explained in NSE4. FortiGate can translate the source IP address of an outgoing connection (source NAT or SNAT). Three different features can be used for that purpose: - Policy NAT: Overload NAT of many to one - IP pool: Four types of IP pools – Overload, one-to-one, fixed port range and port block allocation - Central NAT table FortiGate can also translate the destination IP address of an incoming connection (destination NAT or DNAT). Two different features can be used for that purpose: - Virtual IP (VIP) - Load Balancer
FortiGate III Student Guide
177
DO NOT REPRINT © FORTINET
Firewall Policies
For policy NAT and overload IP pool, the number of available NAT connections depends on four variables: - IP protocol - NAT IP address - Destination IP address - Destination port For each NAT IP address, there are up to 60,416 TCP and 60,416 NAT UDP ports available. For each TCP or UDP connection, two indexes are created to match the traffic and determine the NAT IP address and port. We will see examples of these indexes in the next slides.
FortiGate III Student Guide
178
DO NOT REPRINT © FORTINET
Firewall Policies
(slide contains animation) The slide shows two clients and two servers. There is a FortiGate in the middle doing NAT of the subnet 172.16.1.0/24 to the IP address 10.10.1.254. So, if there is a connection, for example, from 172.16.1.1 to the HTTP server 10.10.1.1, the index that identifies the outgoing traffic includes this information: Source IP: 172.16.1.1 (client IP). Destination IP: 10.10.1.1 (server IP). Protocol: TCP. Source port: 10000 (random source port used by the client). Destination port: 80 (HTTP port number) The second index identifies the incoming traffic: Source IP: 10.10.1.1 (Server IP). Destination IP: 10.10.1.254 (NAT IP). Protocol: TCP. Source port: 80 (HTTP port number). Destination port: 46372 (NAT port selected by the FortiGate) (click) If there is now a connection from 172.16.1.2 to the HTTPS server 10.10.1.2, the first index is: Source IP: 172.16.1.2 (client IP). Destination IP: 10.10.1.2 (server IP). Protocol: TCP. Source port: 25000 (random source port used by the client). Destination port: 443 (HTTPS port number) And the second index is: Source IP: 10.10.1.2 (Server IP). Destination IP: 10.10.1.254 (NAT IP). Protocol: TCP. Source port: 443 (HTTPS port number). Destination port: 46372 (NAT port selected by the FortiGate) In this case is that, as the server IP addresses and ports are different, the sessions can share the same NAT port. So in both cases the source IP and source port are translated to the same NAT IP and NAT port.
FortiGate III Student Guide
179
DO NOT REPRINT © FORTINET
Firewall Policies
So, although both sessions are using the same NAT IP and NAT port, the FortiOS can still use the indexes to uniquely identify each session. FortiOS only has to ensure that the chosen NAT port combined with the other four variables are unique to identify the session.
FortiGate III Student Guide
180
DO NOT REPRINT © FORTINET
Firewall Policies
(slide contains animation) So, let see now the case of two users (172.16.1.1 and 172.16.1.2) connecting to the same HTTP server (10.10.1.1). The FortiGate is again translating 172.16.1.0/24 to 10.10.1.254. The second index (incoming traffic) for the first client contains this information: Source IP: 10.10.1.1 (Server IP) Destination IP: 10.10.1.254 (NAT IP) Protocol: TCP Source port: 80 (HTTP port number) Destination port: 46372 (NAT port selected by the FortiGate) (click) As the second client is using the same NAT IP and going to the same destination IP and TCP port number, the FortiGate must use a different NAT port. The same NAT port cannot be shared by both sessions. So, the second index for the second client includes: Source IP: 10.10.1.1 (Server IP) Destination IP: 10.10.1.254 (NAT IP) Protocol: TCP Source port: 80 (HTTP port number) Destination port: 34020 (different NAT port selected by the FortiGate) In other words, the source IP, destination IP, protocol and destination port are the same. So, FortiGate must use different NAT ports to differentiate each session.
FortiGate III Student Guide
181
DO NOT REPRINT © FORTINET
Firewall Policies
We can calculate the theoretical maximum number (without exhausting the number of NAT ports available) of simultaneous connections for one NAT IP and one destination IP. If we multiply the number of NAT IP addresses (1 in our case), by the number of NAT ports per NAT IP (60,416), by the number of protocols (2, TCP and UDP), by the number of destination IP addresses (1 in our case) and by the maximum number of destination ports (65,535), we get almost 8 billion possible connections.
FortiGate III Student Guide
182
DO NOT REPRINT © FORTINET
Firewall Policies
Although that high number is theoretically possible, in practice, real numbers are much smaller. One reason is that servers never use all 65,535 possible ports. Indeed, most of the connections to a server go to the same port number and IP protocol number. For example, all the connections to a HTTP server are TCP and usually go to port 80 only.
FortiGate III Student Guide
183
DO NOT REPRINT © FORTINET
Firewall Policies
So, if we calculate again the maximum number of simultaneous connections, this time considering one IP protocol number, and one destination port number, we get 60,416. This is a more realistic number.
FortiGate III Student Guide
184
DO NOT REPRINT © FORTINET
Firewall Policies
Some network applications do not support source port address translation. For those cases, enable Fixed Port so the FortiGate translates only the source IP address and not the source port.
FortiGate III Student Guide
185
DO NOT REPRINT © FORTINET
Firewall Policies
If at one point the FortiGate does not have any available NAT port for a new connection, the traffic is rejected and a log is generated.
FortiGate III Student Guide
186
DO NOT REPRINT © FORTINET
Firewall Policies
This part of this lesson is about session helpers and application layer gateways.
FortiGate III Student Guide
187
DO NOT REPRINT © FORTINET
Firewall Policies
To understand what a session helper does, let’s see an example of a network protocol that might have problems when there is a network device doing NAT. The example that we will see is the FTP protocol, specifically, when working in active mode. Any FTP file transfer is composed of two TCP sessions: one for the control channel and one for data transfer. The control channel is always initiated from the client to the server and is used to send the different FTP commands. The FTP commands allow the client to move through the different server folder, specify the type of file transfer and initiate the data channel for uploading or downloading a file. FTP has two modes: active and passive. The mode determines who initiate the data channel. In the case of passive mode, the data channel is initiated by the client. In the case of active mode, the client sends the port command through the control channel. The command includes the client IP address and the TCP port for the incoming data channel. Then, it is the server who initiates the TCP session to the IP address and port number specified by the port command.
FortiGate III Student Guide
188
DO NOT REPRINT © FORTINET
Firewall Policies
(slide contains animation) So, active FTP will not work if the control channel crosses a network device doing NAT that does not have a session helper. In this example a FTP client is connecting to an active-mode FTP server. There is a router in the middle doing NAT of the client IP address 10.0.1.10 to the NAT IP address 10.200.1.1. (click) After the control channel is up, the client sends the port command with its IP address 10.0.1.10 as the destination for the data channel. When that FTP packet crosses the router, the source IP address in the IP header is changed from 10.0.1.10 to 10.200.1.1. However the IP address in the FTP port command is not translated to 10.200.1.1. (click) Once the server receives that FTP command, it tries to bring up the TCP session for the data channel. It sends the SYN packet to the IP address 10.0.1.10, which is most probably not routable as it is a private IP behind a device doing NAT. The file transfer fails.
FortiGate III Student Guide
189
DO NOT REPRINT © FORTINET
Firewall Policies
(slide contains animation) The FTP session helper fixes this problem. Let’s replace the router by a FortiGate. This is what the FortiGate session helper does. (click) The packet with the FTP port command arrives to the FortiGate. (click) The FortiGate not only translates the source IP address in the IP header, but the session helper also translates the IP address inside the FTP port command. If the source port is also translated in the TCP header, the session helper also does the same in the port command. (click) Another important function of the session helper is to temporally create an expected session (or pin hole) for the data channel connection that will come from the server. So, that means that the administrator does not need to manually create firewall policies to allow these incoming TCP sessions (which use random TCP ports numbers). The session helper automatically creates the session and opens the door for the incoming connection on the fly. (click) After that, the server connects the data channel to the right IP address 10.200.1.1. (click) That incoming TCP connection is allowed by the expected session previously created by the session helper, even when there is not firewall policy allowing it.
FortiGate III Student Guide
190
DO NOT REPRINT © FORTINET
Firewall Policies
This slide shows a packet capture of the previous FTP traffic before the port command reaches the FortiGate. We see the original client IP address 10.0.1.10.
FortiGate III Student Guide
191
DO NOT REPRINT © FORTINET
Firewall Policies
This other slide shows another packet capture, this time after the port command crosses the FortiGate. The session helper has translated the IP address inside the port command to 10.200.1.1.
FortiGate III Student Guide
192
DO NOT REPRINT © FORTINET
Firewall Policies
(slide contains animation) SIP is another example of a protocol that requires a session helper in a NAT environment. Similarly to FTP, SIP uses control channels and data channels. In the case of SIP, 4 data channels, 2 per each traffic direction, are required for each call. In this slide, there are two SIP phones with the IP addresses 10.0.1.10 and 172.168.100.205. Additionally, a FortiGate is doing NAT of 10.0.1.10 to 66.171.121.44. (click) Once the control channel is up, a SIP phone sends an invite packet with its IP address and port numbers for two of the four data channels. (click) The FortiGate session helper creates two expected sessions, (click) and translates the IP address inside the invite packet to 66.171.121.44. (click) The remote phone sends now an OK packet to the right destination IP address (66.171.121.44). The packets includes the IP address and ports for the other two data channels. (click) The session helper creates two more expected sessions, this time using the information coming in the OK packet. (click) After that, the 4 data channels can be connected through the 4 expected sessions. Again, firewall policies are not needed to allow this traffic.
FortiGate III Student Guide
193
DO NOT REPRINT © FORTINET
Firewall Policies
There is actually a way to list the expected sessions created by the session helpers. In this example, the command lists an expected session to allow traffic from 10.171.121.38 to 10.0.1.10, port TCP 50365.
FortiGate III Student Guide
194
DO NOT REPRINT © FORTINET
Firewall Policies
The debug flow shows the name of the session helper (if any) that is inspecting the traffic. In this case, it is the FTP session helper. Also, for traffic that matches an expected session previously created by a session helper, the debug flow shows the message: 'find an EXP session'.
FortiGate III Student Guide
195
DO NOT REPRINT © FORTINET
Firewall Policies
There are other protocols that, in some circumstances, also require a session helper, for example PPTP, H323 and RSH. We can list the active session helpers by typing the command 'show' after 'config system session-helper'. The output lists the TCP or UDP port numbers that each session helper is listening to. If one of those protocols is using a different port number, you need to change the FortiGate configuration to match it.
FortiGate III Student Guide
196
DO NOT REPRINT © FORTINET
Firewall Policies
For SIP traffic inspection, the FortiGate also includes a feature smarter and more versatile than the SIP session helper. It is the SIP application layer gateway (SIP ALG). The SIP ALG has all the same functions than the SIP helper, but provides more features. Another difference is that the all session helpers run in the kernel. SIP ALG, on the other hands, runs as a user space process.
FortiGate III Student Guide
197
DO NOT REPRINT © FORTINET
Firewall Policies
A FortiGate uses either the SIP helper or the SIP ALG depending on the configuration. The system setting 'default-voip-alg-mode' specifies which one is used when no VoIP profile is applied. If it is set to 'proxy-based' (default), SIP ALG is used. If it is set to 'kernel-helper-based', SIP helper is used. If the SIP traffic matches a firewall policy with a VoIP profile, SIP ALG is always used regardless of the 'default-voip-alg-mode' setting. Fortinet recommends the use the SIP ALG. The SIP helper should be used only as an alternative when, for some reason, the SIP ALG is not working as expected.
FortiGate III Student Guide
198
DO NOT REPRINT © FORTINET
Firewall Policies
These are the commands to change the ports for the SIP ALG. The SIP ALG supports SIP over UDP, SIP over TCP and encrypted (SSL) SIP.
FortiGate III Student Guide
199
DO NOT REPRINT © FORTINET
Firewall Policies
The command 'diagnose sys sip-proxy calls list' displays all the active SIP calls. The command 'diagnose sys sip-proxy calls clear' disconnects all the active SIP calls.
FortiGate III Student Guide
200
DO NOT REPRINT © FORTINET
Firewall Policies
There are two real time debugs that display information about SIP traffic: diagnose debug application im 31 and diagnose debug application sip
FortiGate III Student Guide
201
DO NOT REPRINT © FORTINET
Firewall Policies
The last part of this lesson is about traffic shaping.
FortiGate III Student Guide
202
DO NOT REPRINT © FORTINET
Firewall Policies
Three QoS techniques are available in FortiGate: - Traffic policing - Traffic queuing - Traffic shaping
FortiGate III Student Guide
203
DO NOT REPRINT © FORTINET
Firewall Policies
Traffic policing limits the amount of traffic ingressing or egressing an interface. A packet is allowed only if the traffic rate is below the threshold for the respective traffic direction.
FortiGate III Student Guide
204
DO NOT REPRINT © FORTINET
Firewall Policies
Traffic queueing assigns packets to one of different queues, depending on the ToS field of the IP header. Each queue has different priorities: high, medium and low.
FortiGate III Student Guide
205
DO NOT REPRINT © FORTINET
Firewall Policies
Traffic shaping can: - Guarantee bandwidth - Limit bandwidth - Increase or decrease traffic priority
FortiGate III Student Guide
206
DO NOT REPRINT © FORTINET
Firewall Policies
Two types of traffic shapers: Shared and Per-IP. A shared shaper applies aggregated bandwidth limits to all traffic using that shaper. The scope can be per-policy or for all policies referencing that shaper. A per-ip shaper applies the bandwidth limits to each source IP address.
FortiGate III Student Guide
207
DO NOT REPRINT © FORTINET
Firewall Policies
There are two CLI commands that display the amount of packets dropped by shared traffic shapers: diagnose firewall shaper traffic-shaper list diagnose firewall shaper traffic-shaper stats
FortiGate III Student Guide
208
DO NOT REPRINT © FORTINET
Firewall Policies
For the case of per-ip shapers, the equivalent commands are: diagnose firewall shaper per-ip-shaper list diagnose firewall shaper per-ip-shaper stats
FortiGate III Student Guide
209
DO NOT REPRINT © FORTINET
Firewall Policies
These are the topics covered in this lesson.
FortiGate III Student Guide
210
DO NOT REPRINT © FORTINET
Firewall Authentication
This lesson is about firewall authentication.
FortiGate III Student Guide
211
DO NOT REPRINT © FORTINET
Firewall Authentication
During this lesson you will learn to monitor the authentication status of the network users. You will also learn to troubleshoot the most common problems related with local, LDAP and RADIUS authentication.
FortiGate III Student Guide
212
DO NOT REPRINT © FORTINET
Firewall Authentication
Let’s start with some general authentication troubleshooting commands, tools and tips.
FortiGate III Student Guide
213
DO NOT REPRINT © FORTINET
Firewall Authentication
As in the case of any other type of issue, the first step in troubleshooting authentication problems is to get the specifics: Is the problem happening with more than one user? Is the problem really related with user authentication and not with user permissions? The first place to check is usually the logs. What do they show? Is the username correct? Is the traffic being blocked after the authentication? What is the user profiled assigned to the user? In the case of remote server authentication (such as RADIUS or LDAP), check also the logs in the server side. In those cases the result of the FortiGate authentication depends on the server response. If the RADIUS credentials are invalid, what do the RADIUS server’s logs show? Is the user account still active in the RADIUS server?
FortiGate III Student Guide
214
DO NOT REPRINT © FORTINET
Firewall Authentication
This is sample of the user authentication logs. A log can be generated each time a user authentication action is successful or fails.
FortiGate III Student Guide
215
DO NOT REPRINT © FORTINET
Firewall Authentication
You can check the list of active users from either the user monitor in the GUI or from the CLI. The CLI command for that purpose is 'diagnose firewall auth list'. You can previously filter the output with the command 'diagnose firewall auth filter'.
FortiGate III Student Guide
216
DO NOT REPRINT © FORTINET
Firewall Authentication
Any session for traffic coming from an authenticated user contains the authed flag. Additionally, the username and user group is added to the session information.
FortiGate III Student Guide
217
DO NOT REPRINT © FORTINET
Firewall Authentication
There is a real time debug for troubleshooting problems related with any user authentication method: local, remote or even FSSO. It is 'diagnose debug application authd'.
FortiGate III Student Guide
218
DO NOT REPRINT © FORTINET
Firewall Authentication
The second part of this lesson is about debug commands and tools for troubleshooting LDAP authentication problems.
FortiGate III Student Guide
219
DO NOT REPRINT © FORTINET
Firewall Authentication
Let’s start with a quick review of what we learned about the LDAP protocol in NSE4. The hierarchy of an LDAP schema is not required to hold any resemblance to the organization. However, generally the name conventions used and the group structure match with the name of the company and corporate hierarchy very closely. On the top, we have the root or DC. This is where an LDAP tree always starts, with any schema. After that, the groups (or branches) are defined using C, OU, and/or O. The exact behavior and options used depend on the schema and what exactly is being defined. Branches may contain objects and each object contains attributes. Objects are uniquely identified by their distinguish names (DNs). The full DN specifies where the object is and the name and value of an attribute that can be used to find it.
FortiGate III Student Guide
220
DO NOT REPRINT © FORTINET
Firewall Authentication
(slide contains animation) There are 3 different methods, or bind types, for a FortiGate to access a LDAP server: simple, anonymous and regular. During this lesson we will talk only about regular bind, which is the most complex, versatile and commonly used method. LDAP authentication using regular bind does four steps: First, the FortiGate logs into (bind) the LDAP server using a LDAP administrator account. (click) At this point the FortiGate knows only the user name, but it doesn't know the branch where the user is located. So during the second step, the FortiGate does a search query in the LDAP database to find where the user is located. In other words, to find the user’s DN. If the user is found, the server replies with the user’s DN. After that the FortiGate logs out (unbind) from the LDAP server. (click) The third step is another bind, but this time using the user credentials. The FortiGate sends the DN, learned in the step before, together with the password. (click) The last step is to get the user group information. How this work depends on the type of LDAP server, but it is generally a LDAP query. When isolating a LDAP problem, we must find first which of these four steps is failing.
FortiGate III Student Guide
221
DO NOT REPRINT © FORTINET
Firewall Authentication
Most of the LDAP authentication problems are caused by misconfigurations. So, let’s see how to quickly check if our regular-bind LDAP configuration is ok. We will analyze the case of an LDAP server based on Windows AD, which is the most common case. Misconfigurations usually happen in one of these LDAP settings: - Common name identifier - Distinguished name - User DN - Password
FortiGate III Student Guide
222
DO NOT REPRINT © FORTINET
Firewall Authentication
How do you check if the distinguished name is ok? You can run either of these two commands in the Windows AD server’s command prompt: dsquery user –name dsquery user –samid The output displays the user DN. The distinguished name setting specifies a parent branch under which all users are located. The FortiGate searches users in any sub-branch below this parent branch. For the case in this slide, we can set the distinguished name setting, for example, to: dc=tac,dc=ottawa,dc=fortinet,dc=com
FortiGate III Student Guide
223
DO NOT REPRINT © FORTINET
Firewall Authentication
The user DN (or bind DN) setting is the full DN of the LDAP admin account. We can use the same Windows LDAP server command (dsquery) to find that information.
FortiGate III Student Guide
224
DO NOT REPRINT © FORTINET
Firewall Authentication
You can simply copy and paste the full DN output from the server command prompt to the FortiGate configuration.
FortiGate III Student Guide
225
DO NOT REPRINT © FORTINET
Firewall Authentication
(slide contains animation) This is a summary of how to properly configure regular bind for Windows AD. A different type of LDAP server might require a different approach. First the common name identifier is usually either cn or sAMAccountName. If you set it to CN, users must authenticate using their full names (ex: John Smith). If you set it to sAMAccountName, users must authenticate using their login names (ex: jsmith). (click) You can get the distinguished name by querying the users DNs with the Windows AD command dsquery. (click) You can get the user DN by querying the admin DN with the same Windows AD command dsquery. (click) Finally, the password setting is the LDAP administrator’s password.
FortiGate III Student Guide
226
DO NOT REPRINT © FORTINET
Firewall Authentication
The CLI includes a LDAP authentication test command. It is 'diagnose test authserver ldap'. If the credentials are ok, and if the LDAP configuration is ok, you get not only an authentication confirmation, but also a list of the user groups for that user. You can run this test command as soon as you have completed you LDAP server configuration, even before any user group, or authentication firewall policy have been added to the FortiGate. It tests only the LDAP server configuration and the LDAP communication between the FortiGate and the server.
FortiGate III Student Guide
227
DO NOT REPRINT © FORTINET
Firewall Authentication
(slide contains animation) There is a real time debug for LDAP and RADIUS. This is a sample of the output if everything is ok. A start_search_dn message indicates that the FortiGate is doing the second step mentioned earlier: searching for the user in the LDAP tree. The message includes the “base” branch (distinguished name setting) and the name of the attribute used to locate the user (common name identifier setting). (click) If the LDAP server finds the user, the output shows Found DN, follow by the user full DN.
FortiGate III Student Guide
228
DO NOT REPRINT © FORTINET
Firewall Authentication
(slide contains animation) After that, the FortiGate does the third step: biding using the user credentials. (click) Finally, the output shows the step four: getting the list of user groups.
FortiGate III Student Guide
229
DO NOT REPRINT © FORTINET
Firewall Authentication
If there is any problem with either step 1 (admin bind) or step 3 (user bind) you can sniffer the traffic between the FortiGate and the LDAP server to get the error code. They are very explicit about the reason why the biding is failing.
FortiGate III Student Guide
230
DO NOT REPRINT © FORTINET
Firewall Authentication
Let's see the most common problems and how to identify them by the output of the real time debug. If you see the error invalid credentials right after an authentication attempt, this means a problem in step 1. There is probably an issue with the admin account credentials.
FortiGate III Student Guide
231
DO NOT REPRINT © FORTINET
Firewall Authentication
The message No more DN left indicates a problem in step 2. The LDAP server couldn't find the user in the LDAP tree.
FortiGate III Student Guide
232
DO NOT REPRINT © FORTINET
Firewall Authentication
The message auth denied after finding the user indicates a problem in step 3. Either the user credentials are wrong, or the user account is not active.
FortiGate III Student Guide
233
DO NOT REPRINT © FORTINET
Firewall Authentication
Finally, the error: get_member_of_groups-attr= found 0 values indicates a problem in step 4. The user credentials are ok, but no user group information was found. In some LDAP implementations, the user group information is not used. All LDAP users have the same privileges regardless of their AD group memberships. In those implementations, this error can be ignored.
FortiGate III Student Guide
234
DO NOT REPRINT © FORTINET
Firewall Authentication
So, what to do if the problem is in step 1 (admin bind not working)? - Use the dsquery query to check the admin DN - Check the admin password - Sniffer the error code coming from the server What to do if the problem is in step 2 (LDAP server could not find the user): - If the common name identifier is set to sAMAccountName, the user must use the login name. If it is set to cn instead, the user must use the full name. - Check the distinguished name setting with the dsquery command.
FortiGate III Student Guide
235
DO NOT REPRINT © FORTINET
Firewall Authentication
As FortiGate RADIUS configuration is simpler than LDAP configuration, the troubleshooting is also usually simpler. Let’s see quickly how to troubleshoot RADIUS problems.
FortiGate III Student Guide
236
DO NOT REPRINT © FORTINET
Firewall Authentication
Normal authentication queries with the RADIUS protocol begin with an "Access-Request" being sent from the FortiGate to the RADIUS server. Valid responses to this are "Access-Accept" and "AccessReject" (yes and no effectively). If two-factor authentication is enabled on the server, it will come back with an "Access-Challenge" message, where it is essentially looking for more information.
FortiGate III Student Guide
237
DO NOT REPRINT © FORTINET
Firewall Authentication
Similarly to LDAP, there is a CLI test command for RADIUS. In this case, you must provide not only the credentials for a test user, but also the authentication scheme. FortiGate supports the following RADIUS schemes: chap, pop, mschap and mschap2. Also, as in the case of LDAP, this command tests only the FortiGate RADIUS server configuration. It does not require any user group or firewall policy in the FortiGate configuration.
FortiGate III Student Guide
238
DO NOT REPRINT © FORTINET
Firewall Authentication
RADIUS is either a one-step or two-step process (depending on the use of two-factor authentication). It is not as long as the 4-step process that happens with LDAP regular bind. So, the output of the real time debug is usually shorter.
FortiGate III Student Guide
239
DO NOT REPRINT © FORTINET
Firewall Authentication
These are the topics covered in this lesson.
FortiGate III Student Guide
240
DO NOT REPRINT © FORTINET
FSSO
This lesson reviews FSSO operation and provides tools and tips for troubleshooting FSSO problems.
FortiGate III Student Guide
241
DO NOT REPRINT © FORTINET
FSSO
You will learn to check the connectivity between a FortiGate and a collector agent (CA), and between a CA and the domain controllers (DCs). You will also learn to track user logon events from the DC, to the CA and to the FortiGate. After that, the lesson explains how to list the active FSSO users both in the CA and the FortiGate. Additionally, the lesson covers FSSO troubleshooting.
FortiGate III Student Guide
242
DO NOT REPRINT © FORTINET
FSSO
FSSO operation and configuration is covered in NSE4. So, let's do a review of how this authentication method works.
FortiGate III Student Guide
243
DO NOT REPRINT © FORTINET
FSSO
FSSO enables FortiGate to leverage your network’s existing authentication system for firewall authentication. Once a user logs in, he or she can access other network resources without having to authenticate again. These are the two most common FSSO methods: DC agent and polling. DC agent requires one collector agent. It also requires DC agents installed in all Windows domain controllers. The DC agents detect the login events and "push" this information to the CA. The CA forwards the login events to the FortiGate, together with the workstation IP address and user's group information. The polling mode does not require any DC agent installed. In this mode, the CA frequently polls each DC to get the latest login events. There are three types of polling modes: NetAPI: Polls NetSessionEnum API every 9 seconds. WinSecLog: Polls all security event logs every 3 seconds. WMI: Polls specific security event logs every 3 seconds. These poll intervals are estimated times that depend on the number of servers and network latency. If a standalone CA is used, the 3 types of polling methods are supported. Alternatively, the polling can be done directly from the FortiGate (agentless polling mode), so a standalone CA is not required. However, the FortiGate acting as a CA only supports the WinSecLog polling type. NSE4 covers the advantages and disadvantages of each FSSO method.
FortiGate III Student Guide
244
DO NOT REPRINT © FORTINET
FSSO
This slide shows how the FSSO traffic flows and which ports are used. For the case of polling mode, the polling is done from either the FortiGate or the CA, and in both cases port TCP 445 is used. The communication between the CA and the FortiGate, by default, uses port TCP 8000. The communication between the DCs and the CA uses, also by default, port UDP 8002. FSSO commonly works by identifying each user based on the source IP address of the traffic. Each active FSSO user is associated with one or more IP addresses and one IP address is associated only with one user. This creates conflicts in network using terminal or Citrix servers, where more than one user are sharing the same IP address. For those cases, you can install a terminal server (TS) agent. The TS agent provides the CA and FortiGate with not only the source IP address of each user, but also with the source TCP/UDP ports they are using. So, multiple users sharing the same IP address can be identified by combining the source IP address with the source port. The communication between the TS agent and the CA also uses port UDP 8002.
FortiGate III Student Guide
245
DO NOT REPRINT © FORTINET
FSSO
(slide contains animation) Each time the CA receives a logon event, it checks first if the user is in the ignore list. (click) If it is, the logon event is discarded. After that, the CA checks if the user's group information is still in the cache. (click) If it is not, the CA does either a LDAP or an API query to get the group information from Windows AD. Once the CA knows the user groups, it checks if any of them is included in the list of monitored groups. (click) If none of the groups are included, the logon event is not forwarded to the FortiGate. If at least one group is included, the logon event is forwarded to the FortiGate.
FortiGate III Student Guide
246
DO NOT REPRINT © FORTINET
FSSO
(slide contains animation) An external CA periodically checks if each active FSSO user is still logged on. It does this by polling all known active workstations based on a configurable interval. (click) The way of how this checking is done depends on the FSSO mode. For WMI polling mode, the CA checks the WMI service. For all the other modes, the CA checks the HKEY_USERS hive via remote registry services. (click) If a workstation does not respond to these checks, the CA starts a timer (dead entry timeout interval). After the expiration of that timer, the workstation goes to not verified status and the CA assumes that the user has logged out.
FortiGate III Student Guide
247
DO NOT REPRINT © FORTINET
FSSO
(slide contains animation) We can also configure the external CA to periodically check if the IP address of an active FSSO user has changed. This might happen for example, when a user switches from a wired to a wireless network. When this CA feature is enabled, the CA periodically uses the DNS to resolve the workstation name. (click) If the DNS server replies with a new IP address, the CA sends first a logoff event to the FortiGate, followed by a logon event with the new IP address. So it is important for the DNS server to be immediately and automatically updated each time a user changes the IP address.
FortiGate III Student Guide
248
DO NOT REPRINT © FORTINET
FSSO
There are three important requirements for any FSSO network, and most of the FSSO problems happen when any of these requirements are not fulfilled. As explained, the CA frequently polls each active workstation. For this polling to work properly, TCP ports 139 and 445 must be open between the CA and the workstations. Firewalls must allow this traffic. Additionally, the remote registry service must be up and running on the workstations. As explained in NSE4, DNS is used to get the user IP address. So, administrators must ensure that each workstation name is registered in the DNS server with the right and up-to-date IP address.
FortiGate III Student Guide
249
DO NOT REPRINT © FORTINET
FSSO
We will start now with FSSO troubleshooting.
FortiGate III Student Guide
250
DO NOT REPRINT © FORTINET
FSSO
What steps should an administrator do when troubleshooting a FSSO problem? This slide offers a general checklist. In the next slides, you will learn how to check each of those steps.
FortiGate III Student Guide
251
DO NOT REPRINT © FORTINET
FSSO
If you can reproduce the FSSO problem, there are different tools to track the logon event as it travels from the DC, to the CA and to the FortiGate. Follow these steps: - Perform the login. - Check which DC received the login using the Windows command: echo %logonserver% - Go to that DC and use the Windows event viewer to find the generated logon event. - Check the CA logs and the list of active FSSO users. - Check that at least one user's group is listed as monitored in the group filter. - Go to the FortiGate and check that the logon event was received. List the active FSSO users. Generate traffic from the user workstation and verify that it is added to the FortiGate user monitor.
FortiGate III Student Guide
252
DO NOT REPRINT © FORTINET
FSSO
(slide contains animation) Before starting the troubleshooting, it is recommended to do some changes to the CA logging settings to ensure that all the required debug information will be captured. First, change the log level to Information. (click) Second, increase the log file size limit. If the log file limit is too low, especially in big FSSO networks, you might not have time to see the relevant debug information, as it will be overridden quickly by new log events. (click) Third, you can optionally record the logon events on a different file.
FortiGate III Student Guide
253
DO NOT REPRINT © FORTINET
FSSO
By clicking on show service status in the CA, you can check the connectivity between the FortiGate and the CA. The window displays the serial numbers of all the active FortiGate units.
FortiGate III Student Guide
254
DO NOT REPRINT © FORTINET
FSSO
When a FortiGate is successfully connecting to a CA, the CA logs show these messages. Additionally you can confirm a successful connection while running the FSSO real time debug in the FortiGate.
FortiGate III Student Guide
255
DO NOT REPRINT © FORTINET
FSSO
While the TCP session between the FortiGate and the CA is up, the CA sends heartbeat messages to the FortiGate unit. Both the CA logs and the FortiGate real time debug show this heartbeat interchange.
FortiGate III Student Guide
256
DO NOT REPRINT © FORTINET
FSSO
(slide contains animation) The error "server authentication failed, aborting" in the FortiGate real time debug might indicate a mismatch in the password shared between the CA and the FortiGate. (click) The error "connection refused" might indicate that the TCP communication between the FortiGate and the CA is blocked by a firewall or another device. (click) The error "no route to host" might indicate that the CA IP address is not routable from the FortiGate.
FortiGate III Student Guide
257
DO NOT REPRINT © FORTINET
FSSO
By clicking on the show monitored DCs button on the CA, you can check the communication between the CA and each DC. The table includes the time when the last logon event was received from each DC.
FortiGate III Student Guide
258
DO NOT REPRINT © FORTINET
FSSO
The Event Viewer is a Windows tool that displays all the server logs. You can use it to search for logon event logs.
FortiGate III Student Guide
259
DO NOT REPRINT © FORTINET
FSSO
The FortiGate FSSO real time debug generates messages each time a logon event arrives. They include the name and IP address of the user, together with the workstation name and user's group information.
FortiGate III Student Guide
260
DO NOT REPRINT © FORTINET
FSSO
You can check the list of logon users by selecting show login users. The status OK indicates that the user is active. A user goes to not verified status when either she or he has logged out, or when there is a problem in the polling done by the CA to the workstation.
FortiGate III Student Guide
261
DO NOT REPRINT © FORTINET
FSSO
To get the list of active users from the FortiGate, use the command 'diagnose debug authd fsso list'. You can set up a filter first with the command 'diagnose debug authd fsso filter'.
FortiGate III Student Guide
262
DO NOT REPRINT © FORTINET
FSSO
The CLI command 'diagnose debug authd fsso refresh-logons' refreshes the active FSSO user list in the FortiGate by getting this information again from the CA. The CLI command 'diagnose debug authd fsso clear-logons' flushes the list of active FSSO users in the FortiGate. The CLI command 'diagnose debug authd fsso refresh-groups' refreshes the user group information in the FortiGate by getting this information again from the CA. To list the monitored user groups, use the command 'get user adgrp'.
FortiGate III Student Guide
263
DO NOT REPRINT © FORTINET
FSSO
Agentless polling mode allows the FortiGate to directly poll each DC. The FortiGate acts as a CA, so no standalone CA is required. There are some specific FSSO commands for troubleshooting this mode.
FortiGate III Student Guide
264
DO NOT REPRINT © FORTINET
FSSO
The command 'diagnose debug fsso-polling detail' displays the status and some statistics related with the polls done by the FortiGate to each DC. The command 'diagnose debug fsso-polling refresh-user' flushes the information about all active FSSO users. In agentless polling mode, the FortiGate frequently polls all workstations (as a standalone CA does) to check which users are still logged on. You can sniffer this traffic in port 445.
FortiGate III Student Guide
265
DO NOT REPRINT © FORTINET
FSSO
There is a specific FortiGate daemon that handles the polling mode. It is the fssod daemon. To enable agentless polling mode real time debug use: diagnose debug application fssod -1 The slide shows the most common real time debug errors and the possible causes.
FortiGate III Student Guide
266
DO NOT REPRINT © FORTINET
FSSO
To finish this lesson, we will discuss the most common FSSO problems.
FortiGate III Student Guide
267
DO NOT REPRINT © FORTINET
FSSO
What should you do if the CA does not have logon information? - Verify that the CA is monitoring all DCs - Check that the CA is receiving logon events from the DCs - Test the user account and check the CA logs What should you do if the CA does have the logon information, but the FortiGate does not? - Check that the FortiGate is connected to the CA - Run the real time debugs while testing the user account
FortiGate III Student Guide
268
DO NOT REPRINT © FORTINET
FSSO
If a FSSO user is listed as active in the FortiGate but it cannot browse the Internet, you should first confirm the IP address listed in the FortiGate. Also check the user's group information, firewall policies and CA logs. If the FortiGate is randomly blocking some users after some time, you should check that the CA service, or any FortiGate process, is not crashing. Confirm that the connection between the FortiGate and the CA is up and stable. Try to reproduce the problem and check the CA logs after that.
FortiGate III Student Guide
269
DO NOT REPRINT © FORTINET
FSSO
DNS resolution is essential for FSSO operation. If a CA cannot resolve a workstation name, the user will not be listed as active and the CA logs show the errors in this slide.
FortiGate III Student Guide
270
DO NOT REPRINT © FORTINET
FSSO
Another common problem with FSSO authentication happens when there are applications generating logon events with an AD account different than the users'. In those cases the logon event overrides the information about the user for a workstation IP address (a different user is listed as active for the IP address). The CA is coded to ignore any logon event for anonymous accounts, and account whose name starts with '$' (which are system accounts). If any application is using an account that is overriding the user information, the solution is to add that account to the ignore user list.
FortiGate III Student Guide
271
DO NOT REPRINT © FORTINET
FSSO
Active users with the status not verified are also a common problem. That status is normal after a user has logged out. However, it is not normal for a user that is still logged on, meaning that the polling from the CA to the workstation is failing. In those cases, check the CA logs. They provide more information about the cause of the problem.
FortiGate III Student Guide
272
DO NOT REPRINT © FORTINET
FSSO
What should you do if a user lost internet access after the IP address changed? This type of problem might happen when the user moves back and for from a wired to a wireless network. It might also happens when a workstation gets back from hibernate mode (a gets a new IP address from the DHCP server). The first step is to check the DNS resolution. As explained in a previous slide, when a user changes the IP address, the DNS server must be automatically updated with the new IP information. If that is not possible, one workaround to this problem is to configure FSSO guest access. So, users whose IP addresses have changed can still have some (probably limited) access to some network resources. For the cases where a workstation was assigned more than one IP address (for example one address for the wired network and another one for the wireless), the DNS server should be able to return all those IP addresses when resolving the workstation name. The user is then listed multiple times in the FSSO active user list, one time for each IP address.
FortiGate III Student Guide
273
DO NOT REPRINT © FORTINET
FSSO
These are the topics covered in this lesson.
FortiGate III Student Guide
274
DO NOT REPRINT © FORTINET
IPsec
This lesson is about IPsec troubleshooting. It is one of the most important lessons, as a significant percentage of the support tickets received by Fortinet TAC are related with IPsec.
FortiGate III Student Guide
275
DO NOT REPRINT © FORTINET
IPsec
At the end of this lesson you will know how to bring down, bring up and restart an IPsec VPN. The lesson also teaches how to check the IPsec offloading and use the real time debug to troubleshoot problems during the negotiations of phase 1 and 2. Additionally, you will learn about IPsec traffic capture using the sniffer and debug flow.
FortiGate III Student Guide
276
DO NOT REPRINT © FORTINET
IPsec
(slide contains animation) When isolating IPsec problems, it is useful to understand that an IPsec connection can be described as a 4-step process: (click) 1- "Interesting" traffic triggers the VPN negotiation. Traffic is called "interesting" when it must travel through an IPsec tunnel (encrypted and encapsulated) to reach a remote network. (click) 2- Phase 1 goes up through the negotiation of an IKE security association (SA) (click) 3- One or more phases 2 go up. Two IPsec security association are negotiated per each phase 2 (click) 4- Traffic crosses the tunnel So, if you have an IPsec issue, you should determine in which of these four steps the problem is.
FortiGate III Student Guide
277
DO NOT REPRINT © FORTINET
IPsec
(slide contains animation) We explained the differences between aggressive mode and main mode in NSE4. Let's review those concepts. This shows main mode. 6 packets are exchanged. (click) First, the client initiates by proposing that the tunnel will use one or more security policies. (click) The responder selects which security policy it will agree to use, and replies. (click) Then, the initiator sends its Diffie Hellman public value. (click) The responder replies with its own Diffie Hellman public value. (click) Finally, the initiator sends its peer ID and hash payload, (click) and the responder replies in the same way. (click) As the peer ID is not included in the first packet, it cannot be used by the FortiGate as part of the criteria for selecting the VPN configuration to use. This is not a problem when using point-to-point VPNs as the FortiGate uses the source IP to identify the VPN. It is not a problem either when using one dial-up VPN. But, as we will see later, it can be a problem when using multiple dial-up VPNs.
FortiGate III Student Guide
278
DO NOT REPRINT © FORTINET
IPsec
(slide contains animation) In comparison, let’s show aggressive mode negotiation. Only 3 packets are exchanged: (click) First, the client initiates by suggesting a security policy, and providing its Diffie Hellman public value and peer ID. (click) The responder replies with the same information, plus a hash. (click) Finally, the initiator sends its hash payload. (click) Unlike main mode, the first packet contains the initiator’s peer ID. Therefore, the responder can use this ID as part of the criteria for identifying the VPN configuration to use.
FortiGate III Student Guide
279
DO NOT REPRINT © FORTINET
IPsec
This slide details the selection criteria for dial-up. A FortiGate uses the first VPN configuration (in alphabetical order) that matches: - Local gateway IP - Mode (aggressive or main) - Peer ID, if aggressive mode is used. As explained, only aggressive mode include the Peer ID in the first packet. - Authentication method (pre-shared or PKI) - Digital certificate information, if PKI is used as the authentication method So, if a FortiGate has multiple dial-up VPNs with pre-shared key sharing the same local gateway, you must use aggressive mode and different peer IDs. In this way, the FortiGate identifies the right VPN configuration for each incoming IPsec proposal.
FortiGate III Student Guide
280
DO NOT REPRINT © FORTINET
IPsec
If you need to capture the IPsec traffic, keep in mind that the IP protocol and UDP port numbers depend on NAT-T and the use of NAT. If there is no device in the middle doing NAT, IKE traffic uses UDP port 500 and ESP traffic uses IP protocol 50. The slide shows the two sniffer filters to capture each of those traffic protocols. If, on the other hand, NAT-T is enabled and there is a device in the middle doing NAT, the sniffer command must use a different filter. In this cases IKE traffic starts using port UDP 500, but it switches to UDP port 4500 during the tunnel negotiation. Additionally, ESP traffic is encapsulated inside the UDP-4500 channel.
FortiGate III Student Guide
281
DO NOT REPRINT © FORTINET
IPsec
(slide contains animation) To check the status of the IKE SA, use the command 'diagnose vpn ike gateway list'. It shows, amount other information, when the SA was created, (click) who initiated the VPN, (click) and the time of the last DPD messages sent and received. To manage the IKE SA, use the commands: diagnose vpn ike gateway flush name diagnose vpn ike restart
FortiGate III Student Guide
282
DO NOT REPRINT © FORTINET
IPsec
(slide contains animation) For information about the IPsec SAs (phases 2), use the command 'diagnose vpn tunnel list'. For each phase 2, the output shows: - Gateways IP addresses (click) - DPD counters (click) - NAT-T mode: There are three possible values. none means that there is no device doing NAT between both peers. silent indicates that the remote peer is NATed. keepalive indicates that the local peer is NATed. In both silent and keepalive modes, ESP traffic is encapsulated into UDP packets. (click) - SPIs, negotiated encryption, authentication and keys for each traffic direction (click) - Traffic counters
FortiGate III Student Guide
283
DO NOT REPRINT © FORTINET
IPsec
The same command 'diagnose vpn tunnel' offers options for shutting down, activating or flushing any phase 2.
FortiGate III Student Guide
284
DO NOT REPRINT © FORTINET
IPsec
'get vpn ipsec stats tunnel' shows the number of IPsec VPNs in the configuration. Both 'get vpn ipsec tunnel summary' and 'get ipsec tunnel list' can be used to list summarized information about each IPsec VPN.
FortiGate III Student Guide
285
DO NOT REPRINT © FORTINET
IPsec
(slide contains animation) For detailed information about each tunnel, use the command 'diagnose vpn ipsec tunnel detail'. The output shows traffic counters, (click) negotiated quick mode selectors, (click) and negotiated encryption, authentication and keys.
FortiGate III Student Guide
286
DO NOT REPRINT © FORTINET
IPsec
In some FortiGate models, the encryption and decryption of IPsec traffic can be offloaded to hardware. The supported algorithms depend on the model and type of processor unit doing the offloading. For the cases of NP2, and NP4 with FortiOS 5.2.2 or lower, there is an additional requirement. If IPsec anti-replay is enabled, you must check that IPsec offloading is enabled under 'config system npu'. If the hardware is NP6, NP4lite, or NP4 with FortiOS 5.2.3 or newer, enabling those settings is not required for IPsec offloading. NP2 has an important limitation regarding anti-replay. We will talk about it later.
FortiGate III Student Guide
287
DO NOT REPRINT © FORTINET
IPsec
For the case of IPsec traffic, the FortiGate session table includes a field that describes when the encryption and decryption is being offloaded. First, when the phase 2 goes up, the IPsec SAs are created and loaded to the kernel. As long as there is not traffic crossing the tunnel, the SAs are not copied to the NPU and the npu_flag shows 00. The value of that field also remains in 00 when IPsec offloading is disabled or not supported. Second, when the first IPsec packet arrives, if it is an outbound packet that can be offloaded, the outbound SA is copied to the NPU and the npu_flag changes to 01. If, on the other hands, the first IPsec packet is inbound and can be offloaded, the inbound SA is copied to the NPU and the npu_flag changes to 02. Finally, once both SAs are copied to the NPU, the npu_flag changes to 03.
FortiGate III Student Guide
288
DO NOT REPRINT © FORTINET
IPsec
This command shows number of packets encrypted and decrypted by software (CPU), and by each type of processor unit in the FortiGate.
FortiGate III Student Guide
289
DO NOT REPRINT © FORTINET
IPsec
The IPsec or IKE real time debug is the most useful tool for troubleshooting problems during the tunnel negotiation. Before enabling the debug, you should first set the filter. The recommended filter option is dst-addr4 (or dst-addr6 for IPv6). With this filter, the debug shows only and all the information about the IPsec tunnel whose remote IP address matches the filter.
FortiGate III Student Guide
290
DO NOT REPRINT © FORTINET
IPsec
After setting the filter, enable the IKE real time debug. The table shows what type of output is enabled by each bit in the bit-mask. The most common values for the bit-mask are -1 (all outputs enabled) and 63 (shorter output). They both show the DPD packets and all the information required for troubleshooting IPsec negotiation problems.
FortiGate III Student Guide
291
DO NOT REPRINT © FORTINET
IPsec
(slide contains animation) Let’s see what the real time debug shows under normal circumstances. We analyze the case of main mode first. As explained earlier, main mode requires the interchange of 6 packets, 3 for each traffic direction. The real time debug shows when the first packet (first main mode message) arrives. (click) Then the debug shows the negotiated settings for the phase 1. (click) A message is generated once the FortiGate identifies the VPN configuration to use (with the name of the VPN).
FortiGate III Student Guide
292
DO NOT REPRINT © FORTINET
IPsec
(slide contains animation) The second main mode message arrives. (click) The third main mode message arrives. (click) Pre-shared keys match. (click) And a message is generated to indicate that the phase 1 is up.
FortiGate III Student Guide
293
DO NOT REPRINT © FORTINET
IPsec
(slide contains animation) Let’s see aggressive mode. In this case the negotiation is faster as only three packets are interchanged. After the first aggressive mode message arrives, (click) the debug shows the result of the phase 1 negotiation. (click) Then the VPN is identified. (click) The second aggressive mode message arrives. (click) A message is generated to confirm that the pre-shared keys match. (click) Another message is generated to indicate that the phase 1 is up.
FortiGate III Student Guide
294
DO NOT REPRINT © FORTINET
IPsec
(slide contains animation) Now, let’s see the output for a phase 2 negotiation. The debug shows the phase 2 proposal from the local gateway, (click) and the phase 2 proposal coming for the remote gateway.
FortiGate III Student Guide
295
DO NOT REPRINT © FORTINET
IPsec
(slide contains animation) Then, the output shows the negotiated phase 2 settings. (click) The last messages confirm that the phase 2 is up.
FortiGate III Student Guide
296
DO NOT REPRINT © FORTINET
IPsec
We explained extended authentication (XAuth) in NSE4. Here, you will see what packets are interchanged during extended authentication. XAuth happens after the phase 1 is up and before any phase 2 negotiation. That is why XAuth is also called phase 1.5. In any XAuth communication there is always one client and one server. The server sends a CFG_REQUEST packet, which must be replied by the client with a CFG_REPLY packet. The CFG_REPLY packet includes the user credentials. If the authentication is ok, the server sends CFG_SET and the client replies with CFG-ACK.
FortiGate III Student Guide
297
DO NOT REPRINT © FORTINET
IPsec
The IKE real time debug shows this interchange.
FortiGate III Student Guide
298
DO NOT REPRINT © FORTINET
IPsec
The messages in the real time debug, related with the CFG_REPLY, show the XAuth user and group names.
FortiGate III Student Guide
299
DO NOT REPRINT © FORTINET
IPsec
If a VPN is using XAuth, the output of the command 'diagnose vpn ike gateway list' shows the username. The same information is displayed in the IPsec monitor in the FortiGate GUI.
FortiGate III Student Guide
300
DO NOT REPRINT © FORTINET
IPsec
A FortiGate has two different methods for automatically configuring the IP settings of an IPsec client: DHCP over IPsec and IKE mode configuration. We analyze the DHCP over IPsec method first. After the phase 1 is up, and the extended authentication is completed, the client negotiates a provisional phase 2 of short duration only for the DHCP traffic. Once that provisional phase 2 is up, standard DHCP traffic is interchanged to set the client IP settings: DHCP_DISCOVER, DCHP_OFFER, DHCP_REQUEST and DHCP_ACK. After the client gets the IP settings (IP address, default gateway, DNS, among others) the phase 2 created for DHCP is turned down. After that, a new phase 2 is negotiated for user traffic.
FortiGate III Student Guide
301
DO NOT REPRINT © FORTINET
IPsec
(slide contains animation) The IKE real time debug shows the negotiation of the phase 2 for DHCP. In this first phase 2, the client IP address is used in the quick mode selectors. (click) Then the DHCP phase 2 is deleted, (click) and a new phase 2 is created for use traffic. In this second phase 2, the IP address assigned by the DHCP server is used in the quick mode selectors.
FortiGate III Student Guide
302
DO NOT REPRINT © FORTINET
IPsec
When troubleshooting DHCP over IPsec problem, you can also use the DHCP real time debug. It shows information about the DHCP traffic, including all the IP settings assigned to the IPsec client.
FortiGate III Student Guide
303
DO NOT REPRINT © FORTINET
IPsec
The other method for assigning IP settings to an IPsec client is IKE mode configuration. Similarly to DHCP over IPsec, it happens after the phase 1 and extended authentication and before any phase 2 negotiation. The client sends a CFG_REQUEST message listing the required IP settings (or attributes). The server replies with a CFG_REPLY which contains the assigned values for each of the attributes requested.
FortiGate III Student Guide
304
DO NOT REPRINT © FORTINET
IPsec
The IKE real time debug shows when the CFG_REQUEST and CFG_REPLY packets are sent.
FortiGate III Student Guide
305
DO NOT REPRINT © FORTINET
IPsec
Let's say now that the phase 1 goes up, the phase 2 also goes up, but, for some reason, the traffic is not crossing the tunnel. For those cases, the best troubleshooting tool is the debug flow. If possible, run it in both gateways. It will let you know if the local gateway is dropping the packets, if the local gateway is not routing the packets through the tunnel, or if the remote gateway is the one dropping the packets. This slide shows the normal output. You should see a message indicating that the packet goes to the tunnel (with the tunnel name), and another one explaining that the packet is being encrypted.
FortiGate III Student Guide
306
DO NOT REPRINT © FORTINET
IPsec
We will do now a summary of the most common problems and solutions. If the tunnel is not coming up, use the IKE real time debug. In those cases, you usually get one of this error messages. When the tunnel is bouncing, you usually see DPD packets lost, as an indication that the problem might be on the ISP side.
FortiGate III Student Guide
307
DO NOT REPRINT © FORTINET
IPsec
If the tunnel is up, but traffic does not go through, use the debug flow. It could be that packets are being dropped at either the local or remote gateway. It could be that packets are not being routed properly. Or it could be that packets do not match the quick mode selectors, so they are dropped.
FortiGate III Student Guide
308
DO NOT REPRINT © FORTINET
IPsec
Most of the IPsec problems happen when creating tunnels between FortiGate and a third-party device. Differently than FortiGate, some vendors do not support quick mode selectors that are either 0.0.0.0/0, or use subnets with different sizes. So, they require a different SA (and so a different phase 2) for each pair of local and remote protected subnets. On those cases, the FortiGate configuration might get complicated and long as administrators need to create all the different phases 2. One alternative is to use the 'mesh-selector-type' setting in the phase 1. When it is set to 'subnet' the FortiGate automatically creates the phases 2 on the flight and per demand. This might simplify the FortiGate configuration considerably.
FortiGate III Student Guide
309
DO NOT REPRINT © FORTINET
IPsec
Anti-replay is an IPsec mechanism that adds a sequence number to the ESP headers. The number is incremented each time a packet is sent. It protects the tunnel against replay attacks. We mentioned earlier that there is a restriction related with NP2 and IPsec anti-replay. NP2 cannot properly generate this sequence number. So, if encryption offloading is enabled in a NP2 platform, you might get warning and packets dropped if the remote site has anti-reply enabled. Therefore, you would need to disable either anti-replay or encryption offloading. This limitation does not apply to NP4 and NP6 platforms.
FortiGate III Student Guide
310
DO NOT REPRINT © FORTINET
IPsec
These are the topics covered in this lesson.
FortiGate III Student Guide
311
DO NOT REPRINT © FORTINET
Security Profiles
During this lesson, you will learn to troubleshoot some of the security profiles (or UTM inspection) features.
FortiGate III Student Guide
312
DO NOT REPRINT © FORTINET
Security Profiles
You will learn to troubleshoot FortiGuard, web filtering and antivirus issues. You will also learn to monitor the IPS and fix certificate warning problems during full SSL inspection. Additionally, the lesson compares proxy-based and flow-based inspection modes and describes the FortiGate packet inspection steps.
FortiGate III Student Guide
313
DO NOT REPRINT © FORTINET
Security Profiles
The first part is about FortiGuard.
FortiGate III Student Guide
314
DO NOT REPRINT © FORTINET
Security Profiles
The FortiGate GUI can be used to quickly check the status of the communication between FortiGate and FortiGuard. A green checkmark beside each FortiGuard service indicates that FortiGate can reach FortiGuard and that the license for that service is valid.
FortiGate III Student Guide
315
DO NOT REPRINT © FORTINET
Security Profiles
(slide contains animation) To learn how to troubleshoot FortiGuard problems, we need to understand first how FortiGuard communication works. Also, the communication between FortiGate and FortiGuard for web filtering and antispam is different than in the case of antivirus and IPS. Let's see how FortiGuard web filtering and antispam works first: Initially, FortiGate contacts the DNS server to resolve the name service.fortiguard.net. FortiGate should get a list of IP addresses for servers (usually two or three) that can be contacted to validate the FortiGuard license. (click) FortiGate contacts one of those servers to check the license and get a list of servers that can be used to submit web filtering and antispam rating queries. (click) FortiGate starts sending rating queries to one of the servers in the list. We will explain soon how FortiGate chooses the server. (click) If the chosen server does not reply, FortiGate uses the next server in the list.
FortiGate III Student Guide
316
DO NOT REPRINT © FORTINET
Security Profiles
For web filtering and antispam, the communication between FortiGate and FortiGuard uses either port 53 or 8888. The GUI also offers a test button to check this connectivity.
FortiGate III Student Guide
317
DO NOT REPRINT © FORTINET
Security Profiles
The command 'diagnose debug rating' shows the list of servers for web filtering and antispam queries. For each IP address, the table shows: - The round trip delay - The server's time zone - The amount of recent and consecutives queries without reply - The historical total amount of queries without reply
FortiGate III Student Guide
318
DO NOT REPRINT © FORTINET
Security Profiles
This is how FortiGate selects the server from the list to send the rating requests. FortiGate initially uses the delta between the server's time zone and the FortiGate's system time zone multiplied by 10. This is the server initial weight. The weight goes up with each packet lost. The weight goes down over time if there are no packets lost. But, it never goes below the initial value. FortiGate uses the server with the lowest weight as the one for the rating queries.
FortiGate III Student Guide
319
DO NOT REPRINT © FORTINET
Security Profiles
The output of the command 'diagnose debug rating' shows some flags beside some of the servers: I: This was the server initially contacted to validate the license and get the server list. Usually, there is only one server with this flag D: FortiGate got this IP address when resolving the name service.fortiguard.net. If the administrator has not overwritten the FortiGuard FQDN or IP address in the FortiGate configuration, there are usually two or three servers with this flag S: FortiGate got this IP address from a FortiManager T: Server is not replying to FortiGate queries F: Server is down
FortiGate III Student Guide
320
DO NOT REPRINT © FORTINET
Security Profiles
In many cases, problems related with FortiGuard are caused by ISPs. Some ISPs block traffic in port 53 that is not DNS, or that contains big-size packets (as FortiGuard does). In those cases the solution is to switch FortiGuard traffic from port 53 to port 8888. Some other ISPs (or upstream firewalls) block traffic to port 8888. In those cases the solution is to use port 53. There are also a few cases where ISPs block traffic based on source ports. Changing the source port range for FortiGuard to the one in this slide usually fixes the issue.
FortiGate III Student Guide
321
DO NOT REPRINT © FORTINET
Security Profiles
Let's move to FortiGuard antivirus and IPS. For these two services the communication between FortiGate and FortiGuard happens much less frequently. In the case of web filtering and antispam, FortiGate goes to FortiGuard each time it needs to rate a web site or email (if the information is not in the FortiGate cache). In the case of antivirus and IPS, FortiGate goes to FortiGuard usually once a day to check and download any new version of the AV or IPS databases and engines. This is done using port TCP 443. By default, FortiGate connects to update.fortiguard.net. If FortiGate must connect through a web proxy, these are the CLI commands to use. Usually, clients connecting to a web proxy do not contact the DNS server to resolve names, as it is the web proxy who does it. But, in the case of FortiGuard, FortiGate always requires DNS access, even when connecting through a web proxy.
FortiGate III Student Guide
322
DO NOT REPRINT © FORTINET
Security Profiles
Both the AV and IPS databases can be updated either automatically or manually. Automatic updates download the portions of the databases that have changed since the last update. Manual updates download the whole database if there is new version available. In a few cases, FortiGuard problems are fixed by doing a manual update. This forces FortiGate to download and install the whole database. For example, if the AV database is corrupted, a manual update might fix the problem.
FortiGate III Student Guide
323
DO NOT REPRINT © FORTINET
Security Profiles
The command 'diagnose test application dnsproxy 7' displays the FQDN and IP addresses of the FortiGuard servers available for AV and IPS updates. The command 'diagnose autoupdate status' provides a summary of the FortiGuard configuration in the FortiGate.
FortiGate III Student Guide
324
DO NOT REPRINT © FORTINET
Security Profiles
'diagnose autoupdate versions' lists all the FortiGuard databases and engines installed. The information includes the version, contract expiration date, time it was updated and what happened during last update. The output displays information about the AV engine, AV database and IPS database.
FortiGate III Student Guide
325
DO NOT REPRINT © FORTINET
Security Profiles
It also shows the IPS engine, device identification database and IP geography database.
FortiGate III Student Guide
326
DO NOT REPRINT © FORTINET
Security Profiles
If there are problems updating the AV or the IPS, or if there are problems validating the license, you can use the FortiGuard real time debug: diagnose debug application update -1 diagnose debug enable After enabling the debug, you can force a manual update from the CLI with the command: execute update-now For FortiGuard web filtering and antispam, the commands for enabling the real time debugs are different. We will see the web filter real time debug later.
FortiGate III Student Guide
327
DO NOT REPRINT © FORTINET
Security Profiles
Remember that FortiGuard traffic originates always from the management VDOM. So, the management VDOM (which is root by default) must have Internet access. Proper DNS access from the management VDOM is also important. The FortiGate must be able to resolve the names: update.fortiguard.net service.fortiguard.net Also, keep in mind that updates to FortiGuard contracts are NOT instantaneous. It usually takes around 1 or 2 hours to update a contract in all the servers. However, in some cases, it could take up to 24 hours. So, if you have just changed or renewed your FortiGuard contract and you do not see the change in the FortiGate, most probably you just need to wait a bit more, to give FortiGuard time to synchronize the information in all the servers.
FortiGate III Student Guide
328
DO NOT REPRINT © FORTINET
Security Profiles
Before talking about how to troubleshooting the most common UTM features, let's talk about inspection modes and life of a packet.
FortiGate III Student Guide
329
DO NOT REPRINT © FORTINET
Security Profiles
Proxy and flow-based inspection modes are covered in NSE4. Let's do a quick review. Most of the UTM features (AV, web filtering) can work in either proxy or flow-based mode. In proxy-based mode, the FortiGate transparently proxies any TCP session between a client and a server. So, actually two TCP sessions are created: one between the client and the FortiGate, and another one between the FortiGate and the server. This mode is usually slower than flow-based, but it offers all the functionality.
FortiGate III Student Guide
330
DO NOT REPRINT © FORTINET
Security Profiles
Flow-based mode, on the other hand, does not proxy TCP sessions. The traffic is inspected as it flows through the unit as a single TCP session. This mode is usually faster that proxy-based, but offers less functionality.
FortiGate III Student Guide
331
DO NOT REPRINT © FORTINET
Security Profiles
If traffic matches a firewall policy that combines proxy-based with flow-based inspection profiles, FortiGate does proxy-based inspection for all the profiles that support both modes. This does not apply to IPS or application control profiles, which are always and only flow based.
FortiGate III Student Guide
332
DO NOT REPRINT © FORTINET
Security Profiles
Application proxies are used during proxy-based inspection. They split the TCP session into two and decide what actions to take. They also buffer the traffic and files, generates logs and archives information.
FortiGate III Student Guide
333
DO NOT REPRINT © FORTINET
Security Profiles
Two session flags indicate if the traffic is being inspected in either proxy or flow-based mode. The flag redir means proxy-based mode. The flag ndr means flow-based mode. Also, and for the case of proxy-based inspection, the debug flow shows the message "send to application layer".
FortiGate III Student Guide
334
DO NOT REPRINT © FORTINET
Security Profiles
During the following slides we are going to analyze how a packet is processed by FortiGate since arrives to the unit until egresses the unit. We will see what actions the FortiGate takes and the order of those actions. We have split the whole process into four stages. The first stage happens when the packet is ingressing: First, you can set a limit in the incoming bandwidth ('inbandwidth' parameter). If the traffic has exceeded that limit, the packet is drop. After that, FortiGate checks the thresholds in the DoS policies. The next steps are the reverse path forwarding and IP header integrity checks. If the traffic terminates at the FortiGate (for example management traffic, web proxy, DNS) the packet is processed by the specific daemon that handles the requested feature. If the traffic does not terminate at the FortiGate, it goes to the second stage: routing and firewall policy.
FortiGate III Student Guide
335
DO NOT REPRINT © FORTINET
Security Profiles
Before doing the routing lookup, the FortiGate (if required) applies the destination NAT. If there is no route to the destination, the packet is dropped. In a similar way, if the packet is denied by a firewall policy, it will not be allowed. If the packet is allowed, FortiGate checks if the policy requires authentication. After those steps, FortiGate proceeds to identify the traffic, which is then inspected by any session helper that might be required.
FortiGate III Student Guide
336
DO NOT REPRINT © FORTINET
Security Profiles
The third stage is UTM inspection. If the traffic is encrypted and full SSL inspection is used, the unit proceeds to decrypt the traffic. After that, the inspection profiles are applied in this order: IPS, application control, VoIP, DLP, antispam, web filtering and antivirus.
FortiGate III Student Guide
337
DO NOT REPRINT © FORTINET
Security Profiles
After inspection, FortiGate applies traffic shaping and source NAT. Finally, if the packet must be routed through an IPsec VPN, it is encrypted before egressing the unit.
FortiGate III Student Guide
338
DO NOT REPRINT © FORTINET
Security Profiles
We will talk now about the most common UTM problems. We will start with antivirus.
FortiGate III Student Guide
339
DO NOT REPRINT © FORTINET
Security Profiles
This is the order of how antivirus inspection is executed. First, the unit does the virus scan, then the grayware scan, and finally the heuristic scan.
FortiGate III Student Guide
340
DO NOT REPRINT © FORTINET
Security Profiles
One of the best tools for troubleshooting antivirus issues is the FortiGate logs. This is a sample of a log generated when the FortiGate detects a virus. You can see the name of the file, time and name of the virus.
FortiGate III Student Guide
341
DO NOT REPRINT © FORTINET
Security Profiles
What should an administrator do if a virus is detected in a workstation behind a FortiGate doing antivirus. Again, the first step is to get specific information: What AV software did detect the virus? What is the name of the virus? How did it get inside the network? It might have got inside, for example, through a CD or USB stick, so the infected file never went through the FortiGate.
FortiGate III Student Guide
342
DO NOT REPRINT © FORTINET
Security Profiles
Test if the FortiGate antivirus is properly inspecting the user traffic. Go to eicar.org from a user workstation and try to download the EICAR sample file. If you have a sample of the virus, submit it to FortiGuard and check if it is present in any of the FortiGate virus databases. If it is, check the name of the AV database where it is present. Does your FortiGate have that database installed?
FortiGate III Student Guide
343
DO NOT REPRINT © FORTINET
Security Profiles
There are three FortiGate AV databases: normal, extended and extreme. Not all the models support the 3 databases, and not all the databases might be installed in your FortiGate device.
FortiGate III Student Guide
344
DO NOT REPRINT © FORTINET
Security Profiles
Also remember the restrictions for antivirus inspection: - SSL traffic requires SSL deep inspection - Archives files, such as ZIP files, are examined to certain limits, such as the maximum number of subdirectories and nested archives - Password protected archives cannot be scanned
FortiGate III Student Guide
345
DO NOT REPRINT © FORTINET
Security Profiles
The next part of this lesson is about web filtering.
FortiGate III Student Guide
346
DO NOT REPRINT © FORTINET
Security Profiles
During web filtering inspection, FortiGate first checks the static URL filter list, then the FortiGuard categories and content filtering. Finally, the web filtering can execute some advanced options.
FortiGate III Student Guide
347
DO NOT REPRINT © FORTINET
Security Profiles
Similarly to other UTM features, one of the best troubleshooting tools for web filtering is the FortiGate logs. The unit can generate a log each time a web site is blocked. The log lists the URL, the category, action taken and quota info.
FortiGate III Student Guide
348
DO NOT REPRINT © FORTINET
Security Profiles
Error counters related with web filtering can be listed with the command 'diagnose webfilter fortiguard statistics list'.
FortiGate III Student Guide
349
DO NOT REPRINT © FORTINET
Security Profiles
The output also shows counters for the number of sites allowed and blocked, and statistics about the cache (including hit and miss rates).
FortiGate III Student Guide
350
DO NOT REPRINT © FORTINET
Security Profiles
Another tool for web filtering troubleshooting is the web filter real time debug. You can set a filter first by user (source) IP address with the command: diagnose debug urlfilter src-addr After that, you can enable the real time debug: diagnose debug application urlfilter -1 This slide shows a sample output of the real time debug when the URL to categorize is not in the FortiGuard cache. The two messages display the URL, category, source, destination IP addresses and service.
FortiGate III Student Guide
351
DO NOT REPRINT © FORTINET
Security Profiles
This is another sample of the real time debug. But in this case, the URL category was found in the FortiGuard cache.
FortiGate III Student Guide
352
DO NOT REPRINT © FORTINET
Security Profiles
(slide contains animation) Even after filtering the real time debug by source IP address, the output might still be too much to easily find the URL that you are troubleshooting. In those cases, it might be easier just to search the category information in the FortiGuard cache. To list the content of the FortiGuard web filtering cache, use the command 'diagnose test application urlfilter 3'. For each URL, the output lists its rating by domain name and its rating by IP address. The rating by domain name is the first two digits of the first number from left to right. It is the category ID represented in hexadecimal. (click) The rating by IP address is the first two digits of the second number. It is also the category ID represented in hexadecimal. (click) The command 'get webfilter category' lists all the categories with their respective ID numbers. In this list, the IDs are represented in decimal. (click) So, if you want to find the category name for a URL in the cache, use the first command to list the cache, and convert the ID number from hexadecimal to decimal. Then, use the second command to find the category name for that ID number.
FortiGate III Student Guide
353
DO NOT REPRINT © FORTINET
Security Profiles
The command 'diagnose test application urlfilter 2' clears the web filter cache. The command 'diagnose test application urlfilter 99' restarts the web filtering daemon.
FortiGate III Student Guide
354
DO NOT REPRINT © FORTINET
Security Profiles
Tips for troubleshooting web filtering: - Get the specifics first: What are the URLs that are having the problem? Is it random? Does it happen with all the users? - Check the logs - Isn't the problem caused by a wrong user group configuration? Are the user access privileges right? - Run the real time debug while reproducing the issue
FortiGate III Student Guide
355
DO NOT REPRINT © FORTINET
Security Profiles
We are going to briefly talk about monitoring IPS and DoS operation.
FortiGate III Student Guide
356
DO NOT REPRINT © FORTINET
Security Profiles
IPS logs show information about attacks detected by the IPS. We can use them to see the source and destination IP addresses, and time and name of the attack.
FortiGate III Student Guide
357
DO NOT REPRINT © FORTINET
Security Profiles
'diagnose ips packet status' shows general IPS statistics, including amount of packet inspected (by IP protocol) and counters for each action taken.
FortiGate III Student Guide
358
DO NOT REPRINT © FORTINET
Security Profiles
The IPS test application command can be used to restart the IPS process. If, for any reason, you need to restart the IPS daemon, this is the right way to do it (do not use the command 'diagnose sys kill' to restart it). There is also an option for bypassing the IPS. To "toggle" the bypass status (switch it back and for from enabled to disabled) use the option 5. When IPS bypass is enabled, the FortiGate does not do any kind of IPS inspection over the traffic. All IPS inspection is actually bypassed. This is useful when troubleshooting high CPU problems caused by the IPS engine. In those cases, if after enabling IPS bypassing, the CPU usage is still high, it might indicate a problem in the IPS engine that should be reported to Fortinet TAC.
FortiGate III Student Guide
359
DO NOT REPRINT © FORTINET
Security Profiles
The anomalies (or DoS) logs are aggregated. When several incidents occur together, this reduces the number of log messages. In large attacks, the number of incidents can easily reach 100,000 in a few seconds. Generating a log entry for every packet that matches would completely utilize the CPU. So instead, FortiGate collapses incidents by periodically recording only one message for all of them, and noting the number of incidents. Here, the detection threshold was 50, and the total count is 75. So, FortiGate doesn’t make 24 separate log entries (1 for each incident above 50). It’s just one log message.
FortiGate III Student Guide
360
DO NOT REPRINT © FORTINET
Security Profiles
The command 'diagnose ips anomaly list' lists the statistics for traffic matching any DoS policy. For each IP address and DoS policy, the output displays the expiration time (when the entry will be removed from the DoS table) and the packets per seconds.
FortiGate III Student Guide
361
DO NOT REPRINT © FORTINET
Security Profiles
NSE4 introduces the different SSL inspection modes. Here, we are going to review and expand those concepts.
FortiGate III Student Guide
362
DO NOT REPRINT © FORTINET
Security Profiles
There are two SSL inspection modes: SSL certificate inspection and full SSL inspection. When using SSL certificate inspection, the FortiGate is not decrypting the traffic. It is only inspecting the server digital certificates and the name indication (SNI) field, which are interchanged before the encryption. The FortiGate tries first to get the URL from the SNI field. The SNI field is a TLS extension that contains the complete URL that the user is connecting to. It is supported by most of the modern web browsers. If the SNI field is not present (because the web client may not support it), the FortiGate proceeds to inspect the server digital certificate. It specifically uses the FQDN in the common name (CN) field as the URL to categorize. Usually, but not always, this FQDN matches the site the user is connecting to. But, it is not always accurate. That is why it is only use as an alternative when the first method (SNI) fails. If you see that the FortiGate is wrongly identifying a HTTPS URL, check the CN field in the site's digital certificate. It might be that the client does not support SNI and the FortiGate is getting the wrong URL from the certificate. In those cases the solution would be to use either full SSL inspection, or a client that supports SNI.
FortiGate III Student Guide
363
DO NOT REPRINT © FORTINET
Security Profiles
Full SSL inspection is available only in some FortiGate models. In this case, the device does indeed decrypt (and re-encrypt) the SSL traffic. Under normal circumstances (without full SSL Inspection), encrypted traffic cannot be inspected, as the firewall does not have the key required to decrypt the data. In order to work, the FortiGate must be located in the middle of the communication between the user’s browser and the web site. When the browser connects to the site, the web server sends its certificate, which contains its public key. The FortiGate intercepts the web server certificate and generates a new one on the fly. The new certificate is issued by the CA installed on the FortiGate, which is not and well-known public CA. The FortiGate also generates on the fly a new pair of public and private keys.
FortiGate III Student Guide
364
DO NOT REPRINT © FORTINET
Security Profiles
When enabling SSL Inspection, browsers start displaying a certificate warning each time a user is connecting to a HTTPS site. The reason is that the certificates received by the browsers are now being signed by the FortiGate, which is a CA that the browsers do not trust. There are two ways to fix this warning: The first option is to download the default FortiGate certificate for SSL proxy inspection and install it in all the workstations. The second option is to generate a new SSL proxy certificate from a private CA. In this case, the private CA certificate must still be installed in all the workstations. This is not a limitation in FortiGate, but a consequence of how digital certificates were designed to work.
FortiGate III Student Guide
365
DO NOT REPRINT © FORTINET
Security Profiles
Replacing the certificate on the traffic can cause problems. Some software and servers have specific limitations on the certificates that are allowed to be used. HSTS and PKP are security features designed to detect man-in-the-middle SSL attacks by making sure that any certificate presented when accessing a server resource is signed by a specific CA. If it detects any other CA it will simply refuse to continue the SSL handshake and prevent access to the website. The options available for these are limited. One option is to bypass SSL inspection of that traffic. Another option is to use SSL certificate inspection instead.
FortiGate III Student Guide
366
DO NOT REPRINT © FORTINET
Security Profiles
These are the topics covered in this lesson.
FortiGate III Student Guide
367
DO NOT REPRINT © FORTINET
Explicit Web Proxy
NSE4 covers explicit web proxy configuration. This lesson is about explicit web proxy troubleshooting.
FortiGate III Student Guide
368
DO NOT REPRINT © FORTINET
Explicit Web Proxy
At the end of this lesson you will be able to troubleshoot explicit web proxy issues by checking the status of the proxy users, user traffic, sessions and DNS traffic. You will also learn to use the proxy real time debug.
FortiGate III Student Guide
369
DO NOT REPRINT © FORTINET
Explicit Web Proxy
With explicit web proxy, clients do not send their requests directly to the servers, but to the proxy. The proxy processes the requests and forwards them to the servers. Clients must be explicitly configured with the proxy IP address (or FQDN) and the port that the proxy is listening to. A web proxy can optionally cache the web traffic.
FortiGate III Student Guide
370
DO NOT REPRINT © FORTINET
Explicit Web Proxy
There are 3 methods for configuring the explicit web proxy settings in a browser: Manual, using a PAC file, and using the web proxy autodiscovery protocol (WPAD). WPAD is explained in NSE4.
FortiGate III Student Guide
371
DO NOT REPRINT © FORTINET
Explicit Web Proxy
A PAC file defines when the browser must use a proxy and how to choose it (when more than one proxy is available). If you are not using WPAD, and want to use a PAC file, you must configure the browser with the URL of the HTTP server where the file can be downloaded. Usually, the PAC file is stored in the proxy itself. In the case of FortiGate, by default, the PAC file can be downloaded from the URL: http://:/proxy.pac Both the port and the PAC file name are configurable.
FortiGate III Student Guide
372
DO NOT REPRINT © FORTINET
Explicit Web Proxy
(slide contains animation) What does a PAC file contain? A PAC file is a JavaScript function. When browsers run it, it determines whether the request will be proxied, and what proxy to use. In this example: The PAC file allows any connection to example.com to bypass the proxy (click) Connections to servers in the 10.0.0.0/24 subnet use the proxy named fastproxy.example.com (click) All other requests are made through proxy.example.com
FortiGate III Student Guide
373
DO NOT REPRINT © FORTINET
Explicit Web Proxy
Given that cache consumes system resources, do you want all users to be able to use the cache? You can configure the proxy to allow access only to authenticated users that belong to specific user groups. Authentication can be either based on either source IP address or HTTP session cookies. How should you decide which to use? IP-based authentication requires less memory. However, it should only be used when each user has a different IP address. If your users are sharing the same IP address, use HTTP session-based authentication instead. In this mode, each browser inserts an HTTP cookie in its requests. The cookies identify each user. This method requires slightly more memory because FortiGate must remember all session cookies.
FortiGate III Student Guide
374
DO NOT REPRINT © FORTINET
Explicit Web Proxy
The GUI user monitor shows the active explicit web proxy users. To get that list from the CLI, use the command: diagnose wad user list To clear (de-authenticate) all the explicit web proxy user, use the command: diagnose wad user clear
FortiGate III Student Guide
375
DO NOT REPRINT © FORTINET
Explicit Web Proxy
(slide contains animation) For any explicit web proxy connection (if the content is not in the cache) there are two TCP sessions listed separately in the FortiGate session table: one from the client to the proxy, and another one from the proxy to the server. This is a sample of the session entry created for the first TCP session. It has the local flag as it terminates at the FortiGate. (click) The client IP address is the source, (click) and the proxy is the destination.
FortiGate III Student Guide
376
DO NOT REPRINT © FORTINET
Explicit Web Proxy
(slide contains animation) This is a sample of the second TCP session. The local flag is here too, and this session also terminates at the FortiGate. (click) However, in this case the source IP address is the proxy, (click) and the destination is the server.
FortiGate III Student Guide
377
DO NOT REPRINT © FORTINET
Explicit Web Proxy
The command 'diagnose wad session list' shows a summary of the all the sessions related with web proxy. In this case, each pair of TCP sessions (client to proxy and proxy to server) is displayed as one entry.
FortiGate III Student Guide
378
DO NOT REPRINT © FORTINET
Explicit Web Proxy
FortiOS limits the number of simultaneous explicit web proxy users. This limit is hard coded, applies globally (to all VDOMs) and depends on the model. However, there are no limits in the number of simultaneous sessions per user. If user authentication is not used, each source IP address is treated as a different user.
FortiGate III Student Guide
379
DO NOT REPRINT © FORTINET
Explicit Web Proxy
Before running any explicit proxy debug command, you must enable them with the command: diagnose test application wad 2200 After that, use 'diagnose test application wad 112' to get the maximum number of simultaneous proxy users.
FortiGate III Student Guide
380
DO NOT REPRINT © FORTINET
Explicit Web Proxy
The command 'diagnose test application wad 110' shows the amount of concurrent proxy users. The output also shows the list of users, their IP addresses and the authentication method they are using (either IP-based or session-based). As explained in the previous slide, type first the command 'diagnose test application wad 2200' if you have not done it yet.
FortiGate III Student Guide
381
DO NOT REPRINT © FORTINET
Explicit Web Proxy
(slide contains animation) Keep in mind than in the case of explicit web proxy, the proxy itself is responsible for resolving the web site FQDNs. So, the proxy requires a reliable DNS connection. This commands displays web proxy DNS traffic statistics. It shows the number of DNS requests sent, (click) the number of times that the DNS could not resolve a name, (click) and the number of times that the DNS server did not reply.
FortiGate III Student Guide
382
DO NOT REPRINT © FORTINET
Explicit Web Proxy
(slide contains animation) The way of how you enable the real time debug for web proxy is different than the usual way. First, you can optionally create a filter by adding it to the FortiGate configuration. Multiple URLs can be added to the filter. (click) After that, you must enable the use of the filter previously created with the command 'diagnose wad debug-url enable'. (click) Finally, you enable the real time debug with the commands 'diagnose wad console-log enable' and 'diagnose debug enable'.
FortiGate III Student Guide
383
DO NOT REPRINT © FORTINET
Explicit Web Proxy
In this example we are enabling the real time debug to display information about connections to the URL www.fortinet.com. The debug shows good details about what is happening step-by-step with the proxied traffic. First, you should see the HTTP GET request coming from the client. The output shows part of the HTTP header.
FortiGate III Student Guide
384
DO NOT REPRINT © FORTINET
Explicit Web Proxy
(slide contains animation) After that, the debug shows messages related with the DNS resolution of the FQDN in the URL. (slide) The HTTP GET request is forwarded to the cache first, and then to the server if the content is not in the cache.
FortiGate III Student Guide
385
DO NOT REPRINT © FORTINET
Explicit Web Proxy
If it is not in the cache and the server replies, the debug shows an output similar to this one.
FortiGate III Student Guide
386
DO NOT REPRINT © FORTINET
Explicit Web Proxy
Finally, the reply from the server is forwarded to the client.
FortiGate III Student Guide
387
DO NOT REPRINT © FORTINET
Explicit Web Proxy
If, on the contrary, the content was found in the cache, the output is a bit different.
FortiGate III Student Guide
388
DO NOT REPRINT © FORTINET
Explicit Web Proxy
These are the topics covered in this lesson.
FortiGate III Student Guide
389
DO NOT REPRINT © FORTINET
Operation Modes
During this lesson, we will talk about the two FortiGate operation modes: NAT/route and transparent.
FortiGate III Student Guide
390
DO NOT REPRINT © FORTINET
Operation Modes
At the end of this lesson you will know how to identify the route path for any traffic session, diagnose and troubleshoot routing problems and segment a layer-2 network into multiple broadcast domains.
FortiGate III Student Guide
391
DO NOT REPRINT © FORTINET
Operation Modes
Traditional firewalls and NAT mode FortiGate units are routers. So, each interface has to be in different subnets and each forms different broadcast domains. The FortiGate routes IP packets based on the IP header information, overriding the source MAC address. So, if a client sends a packet to a server connected to a different FortiGate interface, the packet arrives to the server with a FortiGate’s MAC address, instead of the client’s. In the case of transparent mode, FortiGate forwards frames without changing the MAC addresses. When the client receives a packet from a server connected to a different FortiGate interface, the frame contains the server’s real MAC address. A transparent mode FortiGate is a Layer-2 bridge or switch. So, the interfaces do not have IP addresses and all belong (by default) to the same broadcast domain. This means that a transparent mode FortiGate can be installed in a customer network without changing the customer’s IP address plan. A FortiGate can combine VDOMs in NAT mode with VDOMs in transparent mode.
FortiGate III Student Guide
392
DO NOT REPRINT © FORTINET
Operation Modes
The first part of this lesson is about NAT or route mode.
FortiGate III Student Guide
393
DO NOT REPRINT © FORTINET
Operation Modes
A FortiGate is a stateful device, so a lot of information is decided right at the beginning of a session, on the first packets. For any traffic session, usually only two routing lookups are done: one on the first packet sent by the originator, and another one on the first reply packet coming from the responder. After that, all the routing information is written in the FortiGate session table. However, after any change to the routing table, the route information is flushed from the affected entries in the session table. So, additionally routing table lookups are done after to repopulate the session table with the new routing information.
FortiGate III Student Guide
394
DO NOT REPRINT © FORTINET
Operation Modes
How does FortiGate decide routes? FortiGate has multiple routing modules. This diagram shows their logic. First FortiGate searches its policy routes. You can view them with the command 'diagnose firewall proute list'. If there is a match in a policy route and the action is Forward Traffic, FortiGate routes the packet accordingly. If the action is Stop Policy Routing, FortiGate goes to the next table, which is the route cache. You can view that content with the CLI command 'diagnose ip rtcache list'. Finally, FortiGate searches the forwarding information base (FIB). The FIB is generated by the routing process, and is the table used for packet forwarding. Think of the routing table’s purpose as for management, while the FIB is for forwarding. This separation becomes clearer in FortiGate active-active HA. In an HA cluster, both route management and forwarding tables exist on the master FortiGate. But on the slave FortiGate, only the forwarding table (FIB) exists. If there’s no match in any of those tables, FortiGate drops the packet because it is unroutable.
FortiGate III Student Guide
395
DO NOT REPRINT © FORTINET
Operation Modes
When there is more than one route to a destination, this is process for selecting which route to use. First, FortiGate uses the most specific route, which is the one with the longest net mask (smallest subnet). If there are two or more routes with the same longest net mask, the unit selects the ones with the lowest distance. After that, the lowest metric is used as tie breaker for dynamic routes. In the case of static routes, the priority is used instead. If there multiples routes with the same net mask, distance, metric, and priority, FortiGate shares the traffic among all of them. This is what is called equal cost multipath (ECMP), which is supported for static, BGP and OSPF routes.
FortiGate III Student Guide
396
DO NOT REPRINT © FORTINET
Operation Modes
The FortiGate adds a static route to the routing table only if all these requirements are met: 1- The outgoing interface is up 2- There is no other route with the same net mask and lower distance 3- The link health monitor (if configured) is up 4- The next-hop IP address is in the same subnet as one of the IP addresses assigned to the outgoing interface
FortiGate III Student Guide
397
DO NOT REPRINT © FORTINET
Operation Modes
Let's quickly review an important routing concept that was introduced in NSE4: reverse path forwarding check. It protects against IP spoofing attacks and routing loops by checking the route to the source IP address. This check is done only on packets on the original direction (not on reply packets) and only when the session was not created yet, or when the routing table has changed. If the check fails, the packet is dropped and the debug flow shows the error: reverse path check fail, drop
FortiGate III Student Guide
398
DO NOT REPRINT © FORTINET
Operation Modes
(slide contains animation) There are two RPF check modes: Loose (which is the default mode) and strict. In the loose mode, the packet is accepted as long as there is one valid route to the source IP through out the incoming interface. It does not have to be the best route, just a valid one. In this example, the packet from 10.4.0.1 to 10.1.0.1 is accepted because the FortiGate has a valid route (the default route) to 10.4.0.1 through out port2. (click) However, the packet from 172.16.1.1 to 10.1.0.1 is NOT accepted, as there is not any valid route to the IP address 172.16.1.1 through out port3.
FortiGate III Student Guide
399
DO NOT REPRINT © FORTINET
Operation Modes
(slide contains animation) In the case of strict mode, the FortiGate checks that the best route to the source IP address is through the incoming interface. The route not only has to be valid (as in the case of loose mode), but it also has to be the best one. If in the same example, we change the FortiGate configuration from loose mode to strict mode. This is the result: 1- The packet from 172.16.1.1 to 10.1.0.1 is still blocked as there is not route at all to that source IP address through out port3 (click) 2- Now the packet from 10.4.0.1 to 10.1.0.1 is also blocked. There is a valid route to 10.4.0.1 through out port2, but it is NOT the best route to that source IP address. The best route to 10.4.0.1 is through out port3. So, in this case, strict mode accepts traffic from the subnet 10.4.0.0/24 only when port3 is the incoming interface.
FortiGate III Student Guide
400
DO NOT REPRINT © FORTINET
Operation Modes
UTM traffic inspection requires symmetric routing. Traffic both ways must follow the same path. So, FortiGate routes traffic symmetrically. This means that, under some network topologies, FortiGate might not route the return traffic through the best path, but through the path that the originating traffic is using. For that purpose the unit “remembers” the “interface to source” and uses that interface to route the return packets even when a better route using a different interface exists. Let’s see an example in the next slides.
FortiGate III Student Guide
401
DO NOT REPRINT © FORTINET
Operation Modes
(slide contains animation) Let’s analyze this network topology. The “local” network 10.1.0.0/24 has three network devices: a local workstation, a local router and a FortiGate (port1). Also the FortiGate port2 is directly connected to the local router (using the subnet 10.2.0.0/24). There is a remote router connected to FortiGate port3 and, behind that, a remote server (10.4.0.1). So, any traffic destined to the remote server must be routed through the FortiGate. One important detail in this network is that the local workstation default gateway is 10.1.0.254. (click) This means that if you send a ICMP echo request from the local workstation to the remote server, the packet goes to the local router first; then to the FortiGate; then to the remote router; and finally to the destination. When the ICMP packet arrives to the FortiGate, an entry for the originating traffic is created in the unit route cache. This entry contains the interface to source, or the incoming interface where the packet arrived, which in this case is port2.
FortiGate III Student Guide
402
DO NOT REPRINT © FORTINET
Operation Modes
(slide contains animation) Additionally, FortiGate creates an entry in the session table. This entry also contains information about the interface to source. (click) As explained earlier, the FortiGate does a first routing lookup to find the next-hop to destination. That IP address is also stored in the session information. (click) As there is not ICMP echo reply yet, you will notice that the next-hop to source is still undetermined (it is 0.0.0.0). It will be determined with the second routing lookup that happens with the first reply packet.
FortiGate III Student Guide
403
DO NOT REPRINT © FORTINET
Operation Modes
Now, let’s see how the FortiGate routes the return packet. When the FortiGate receives the ICMP echo reply, as there is already a session and a route cache created, it uses the interface to source. So, in this case the unit routes the packet through out port2 towards the local router, even when there is a better route to the destination 10.1.0.1. The FortiGate routing table shows port1 as the best route to 10.1.0.1 (locally connected), but it still uses port2. The objective, as explained, is to keep the traffic flows symmetric. With the first ICMP echo reply, a second entry is added to the route cache, this time for the return traffic.
FortiGate III Student Guide
404
DO NOT REPRINT © FORTINET
Operation Modes
Additionally, and as explained earlier, the unit does a second routing lookup, this time to find the next-hop (or gateway) to source. That IP address is added to the session (where 0.0.0.0 was before).
FortiGate III Student Guide
405
DO NOT REPRINT © FORTINET
Operation Modes
Now, this raises one question: What happens is the traffic is originated from the server side instead? Let’s say that the ping is sent from the remote server to the local workstation. In this case, when the ICMP echo request arrives to the FortiGate, there is not session yet. So, the unit uses the best route to 10.1.0.1, which is through port1. These examples show how a FortiGate might, on some network topologies, route packets to the same destination differently depending on who initiated the session.
FortiGate III Student Guide
406
DO NOT REPRINT © FORTINET
Operation Modes
Something interesting in this last example is the reply traffic. As the local workstation default gateway is 10.1.0.254, the ICMP echo reply goes to the local router first. Then the packet arrives to the FortiGate port2. The result is asymmetric routing as the return traffic is following a different path than the originating traffic. The return packet is arriving to port2 instead of port1 (where the originating traffic was sent). In these cases, the FortiGate accepts this asymmetry, no packets are dropped and UTM inspection is not affected.
FortiGate III Student Guide
407
DO NOT REPRINT © FORTINET
Operation Modes
(slide contains animation) After a change in the routing table, the routing information is removed from the sessions affected by the change. Additionally, related route cache entries are deleted. So, two more routing lookups are done again for the next packets to learn the new routing information and store it in the routing table. Also, RFP check is done again for the first packet in the original direction. This is a sample of a session just after a routing change. The gateways in both directions change to 0.0.0.0/0 and the interfaces to 0, indicating that this information must be learned again. (click) Additionally, the dirty flag is added.
FortiGate III Student Guide
408
DO NOT REPRINT © FORTINET
Operation Modes
However, there is an important exception to this rule. After a routing change, sessions with source NAT keep using the same outbound interface, as long as the old route is still valid. Let's see an example. In this case, the FortiGate is connected to two different ISPs. There is client with the IP address 10.1.0.1/24 connected behind the FortiGate. The FortiGate is doing source NAT of the client traffic to a public IP address that depend on which ISP is using. The unit routing table contains two default routes with the same distance, but different priorities, one default route for each ISP. The route with the lowest priority (port1) is the primary one. When both ISP connections are up, this is route selected by the FortiGate for Internet traffic. So, all sessions to the Internet are created using port1 as the outbound interface.
FortiGate III Student Guide
409
DO NOT REPRINT © FORTINET
Operation Modes
If the administrator increases the priority for port1 to a value higher than the priority in port2, this is what happens: 1- All new sessions start using port2 as it is now the one with the lowest priority 2- However, all the existing sessions keep using port1. The default route through port1, although it is not the best one now, it is still valid and the unit is doing source NAT. Those sessions will keep the old route until they expire If the unit were not doing source NAT, all the existing sessions would switch to port2 after the change.
FortiGate III Student Guide
410
DO NOT REPRINT © FORTINET
Operation Modes
Most of the debug commands for routing are under the 'get router info' tree. Let's see some of them in the next slides.
FortiGate III Student Guide
411
DO NOT REPRINT © FORTINET
Operation Modes
(slide contains animation) The command 'get router info routing-table all' displays all the active routes in the routing table. The left column indicates the source for the route. (click) The first number inside the brackets is the distance, (click) and the second one is the metric. This command doesn't show inactive routes. For example, when two static routes to the same destination subnet have different distances, the one with the lowest distance is active. The one with the highest distance is inactive. So, this command displays only one of them (the active one).
FortiGate III Student Guide
412
DO NOT REPRINT © FORTINET
Operation Modes
If you want to display both the active and the inactive routes, use instead the command: get router info routing-table database In this example the command shows one inactive route at the end. It is inactive because it has a higher distance than the one above.
FortiGate III Student Guide
413
DO NOT REPRINT © FORTINET
Operation Modes
This low-level command shows the actual forward information database (FIB), which is the routing information that the kernel uses to route traffic. All the active routes in the routing table must be also present in the FIB. Additionally, the FIB may contain routes that are not in the routing table and were automatically added by the FortiGate.
FortiGate III Student Guide
414
DO NOT REPRINT © FORTINET
Operation Modes
The route cache contains recently used routing entries in a fast-to-search table. It is consulted before the routing table to speeds up the routing lookup process.
FortiGate III Student Guide
415
DO NOT REPRINT © FORTINET
Operation Modes
This table contains all the IP addresses used by the FortiGate and assigned to FortiGate interfaces.
FortiGate III Student Guide
416
DO NOT REPRINT © FORTINET
Operation Modes
The command 'get router info protocols' gives a quick look at the configuration of all the dynamic routing protocols running in the unit.
FortiGate III Student Guide
417
DO NOT REPRINT © FORTINET
Operation Modes
Now, we will move to transparent mode.
FortiGate III Student Guide
418
DO NOT REPRINT © FORTINET
Operation Modes
In transparent mode, each VDOM forms a separate broadcast domain. Interfaces, by default, don’t. Until you change the initial VDOM configuration, all interfaces, regardless of their VLAN ID, are part of the same broadcast domain. The interface command 'forward-domain' is used to segment the VDOM into multiple broadcast domains. Interfaces with different forward domain IDs belong to different broadcast domains. Interfaces with the same ID belong to the same broadcast domain.
FortiGate III Student Guide
419
DO NOT REPRINT © FORTINET
Operation Modes
(slide contains animation) Here’s an illustration that explains the concept. Let's see first a broadcast with all the interfaces on the same forwarding domain. An ARP request is sent by a single device. It reaches FortiGate through one of the interfaces in the VDOM. (click) Because they all belong to the same forwarding domain, FortiGate then re-broadcasts the packet to all the interfaces, even to interfaces that belong to a different VLAN.
FortiGate III Student Guide
420
DO NOT REPRINT © FORTINET
Operation Modes
(slide contains animation) As we explained, forwarding domains are like broadcast domains. Here’s we have the same network, but this time, VLAN 101 is only on 2 interfaces. Placing those interfaces in a separate forward domain ID (101) segregates them. (click) Traffic arriving to one interface is only broadcasted to interfaces that are part of the same forwarding domain.
FortiGate III Student Guide
421
DO NOT REPRINT © FORTINET
Operation Modes
This debug command lists the MAC address forwarding table in a VDOM operating in transparent mode.
FortiGate III Student Guide
422
DO NOT REPRINT © FORTINET
Operation Modes
These are the topics covered in this lesson.
FortiGate III Student Guide
423
DO NOT REPRINT © FORTINET
External BGP
This lesson is about troubleshooting external BGP issues.
FortiGate III Student Guide
424
DO NOT REPRINT © FORTINET
External BGP
The lesson has three main objectives. First, we will do a quick review of BGP concepts and configuration on FortiGate devices. Second, you will learn to monitor and check the status of a BGP communication. Finally, the lesson covers BGP troubleshooting. BGP is a deep topic more extensive than what is covered here. This lesson shows only the basics about the protocol and the FortiGate to troubleshoot of the most common external BGP issues.
FortiGate III Student Guide
425
DO NOT REPRINT © FORTINET
External BGP
Let's start with a review of basic BGP concepts.
FortiGate III Student Guide
426
DO NOT REPRINT © FORTINET
External BGP
An autonomous system (AS) is a set of routers and networks under the same administration. Each AS is identified by a unique number and usually runs an interior gateway protocol, such as OSPF or RIP.
FortiGate III Student Guide
427
DO NOT REPRINT © FORTINET
External BGP
An external routing protocol (EGP) exchanges routing information between autonomous systems. BGP4, which runs in the Internet, is the dominant EGP protocol today. BGP is typically used when strict control is required over a large number of routes. Two BGP routers exchange AS path information for destination prefixes or subnets. When two routers start a BGP communication, the whole BGP routing table is interchanged. After that, only network updates are sent.
FortiGate III Student Guide
428
DO NOT REPRINT © FORTINET
External BGP
A BGP speaker or peer is a router that sends and receives BGP routing information. The connection between two BGP peers is called a BGP session.
FortiGate III Student Guide
429
DO NOT REPRINT © FORTINET
External BGP
There are three types of autonomous systems: -A Stub AS handles and routes only local traffic and it has only one connection to another AS. One example is a company running BGP, with its own AS number, and one ISP connection -Multi-homed AS also handles and routes only local traffic, but it has multiple connections to different autonomous systems. One example is a company running BGP, with its own AS number, and multiple ISP connections -Transit AS handles and routes not only local traffic, but traffic that is originated and terminates in different autonomous systems (transit traffic). An ISP is an example of a transit AS
FortiGate III Student Guide
430
DO NOT REPRINT © FORTINET
External BGP
A BGP router stores the routing information into three logical tables. The RIB-In table contains all the routing information received from other BGP routers before any filtering. The local RIB table contains that same information after the filtering. The RIB-Out table contains the BGP routing information selected to advertise to other BGP routers.
FortiGate III Student Guide
431
DO NOT REPRINT © FORTINET
External BGP
This flow graph summarizes the BGP process. BGP routes received from other routers are stored in the RIB-In table. A filter is applied and the resulting routes are stored in the local RIB table. Then routes redistributed from the routing table are added, and another filter (outbound) is applied. The resulting routes are the ones being advertised by the BGP router.
FortiGate III Student Guide
432
DO NOT REPRINT © FORTINET
External BGP
BGP routes traffic based on AS paths. Each AS patch includes attributes, which some are used to select the "best" route to each destination. One of the attributes is the AS list, which contains the autonomous systems through which the traffic must pass to reach the destination.
FortiGate III Student Guide
433
DO NOT REPRINT © FORTINET
External BGP
There are four types of BGP attributes: - Well-known mandatory - Well-known discretionary - Optional transitive, which can be passed from one AS to another - Optional non-transitive, which cannot be passed from one AS to another
FortiGate III Student Guide
434
DO NOT REPRINT © FORTINET
External BGP
This is a list of the supported attributes with their types.
FortiGate III Student Guide
435
DO NOT REPRINT © FORTINET
External BGP
Some of the attributes are used during the routing selection process. If all those attributes for multiple routes to the same destination match, and if ECMP is enabled, FortiGate shares the traffic among up to 10 BGP routes. If ECMP is not enabled, the FortiGate uses the route to the router with the lowest BGP router ID.
FortiGate III Student Guide
436
DO NOT REPRINT © FORTINET
External BGP
There are three important considerations related with the way of how BGP is implemented in FortiGate devices. First, there are not hard-code limits. The limitations regarding the number of neighbors, routes and policies depend exclusively on the system memory available. Second, FortiGate by default does not announce any prefix. An administrator must enable redistribution or manually indicate the prefixes to advertise. Third, all prefixes received, by default, are accepted. Administrators can optionally filter out or modify some prefixes.
FortiGate III Student Guide
437
DO NOT REPRINT © FORTINET
External BGP
So, FortiGate BGP, by default, does not advertise any prefix. One way to configure FortiGate to do it is by using the 'redistribution' commands. Connected, static and routes learned from other routing protocols can be redistributed into BGP. You can optionally add route maps to filter the prefixes or modify some of their BGP attributes.
FortiGate III Student Guide
438
DO NOT REPRINT © FORTINET
External BGP
The other way to indicate the prefixes to advertise is by using the 'network' command. It is important to know that, by default, an exact match of the prefix in the 'network' command must be active in the routing table. If there isn't any active route whose destination subnet matches the prefix, FortiGate does not advertise it. This default behavior can be changed by disabling the setting 'network-import-check.' Once disable it, all prefixes in the BGP network table are always advertised, regardless of the actives routes present in the routing table.
FortiGate III Student Guide
439
DO NOT REPRINT © FORTINET
External BGP
This is a sample of a basic FortiGate BGP configuration. In this case the administrator has configured the local AS (65075) and one neighbor with the IP address 10.127.0.117 and AS 65117. The BGP ID of the local router is 0.0.0.75, and the redistribution of static routes into BGP is enabled.
FortiGate III Student Guide
440
DO NOT REPRINT © FORTINET
External BGP
Also by default, FortiGate accepts BGP sessions only from BGP neighbours that are "locally" connected. In other words, neighbours that are in the same subnet (and broadcast domain) than the local FortiGate. There are at least two cases where this requirement creates problems: First, when there are routers between the two BGP peers, in other words, when both peers are not in the same subnet. Second, when a loopback interface is used as the common BGP neighbour IP address for all the BGP peers. For example, a FortiGate running BGP with two or more ISPs, and all the ISPs using the FortiGate loopback IP address as their remote neighbor. In those cases, administrators must enable 'ebgp-enforce-multihop'.
FortiGate III Student Guide
441
DO NOT REPRINT © FORTINET
External BGP
We will move to BGP troubleshooting.
FortiGate III Student Guide
442
DO NOT REPRINT © FORTINET
External BGP
This graph shows the BGP neighbor states and how they change: Idle: Initial state. Connect: Waiting for a successful 3-way TCP connection. Active: Unable to establish the TCP session. OpenSent: Waiting for an OPEN message from the peer. OpenConfirm: Waiting from the keepalive messages from the peer. Established: Peers have successfully interchange OPEN and keepalive messages.
FortiGate III Student Guide
443
DO NOT REPRINT © FORTINET
External BGP
Most of the BGP diagnostic commands are under the 'get router info bgp' tree.
FortiGate III Student Guide
444
DO NOT REPRINT © FORTINET
External BGP
(slide contains animation) This is usually the first debug command we use to get a quick overview of how BGP is working and the status of all neighbors. It shows the local router ID and AS. For each neighbor, the output also displays: Their AS, (click) packet counters, (click) and for how long the neighbor has been up. (click) The last column is the neighbor state and number of prefixes. If the state is not established, this column displays the BGP state. If it is established, this column displays the number of prefixes received by the local FortiGate from that neighbor.
FortiGate III Student Guide
445
DO NOT REPRINT © FORTINET
External BGP
The command 'get router info bgp neighbors' provides detailed information about each BGP neighbor. It includes packet counters, and number of prefixes announced and accepted.
FortiGate III Student Guide
446
DO NOT REPRINT © FORTINET
External BGP
It also shows the number of times that the session has dropped and last time it was reset.
FortiGate III Student Guide
447
DO NOT REPRINT © FORTINET
External BGP
This command provides details about the prefixes being advertised by the local router.
FortiGate III Student Guide
448
DO NOT REPRINT © FORTINET
External BGP
This is the command to display the routes advertised by a neighbor.
FortiGate III Student Guide
449
DO NOT REPRINT © FORTINET
External BGP
These are the commands for enabling and disabling the BGP real time debug.
FortiGate III Student Guide
450
DO NOT REPRINT © FORTINET
External BGP
The next two slides show a sample of the real time debug output during the successful establishment of a BGP session. It shows when the session goes to OpenSent state.
FortiGate III Student Guide
451
DO NOT REPRINT © FORTINET
External BGP
(slide contains animation) When the session goes to OpenConfirm, (click) and then to Established. (click) It also lists the prefixes received once the BGP session is established.
FortiGate III Student Guide
452
DO NOT REPRINT © FORTINET
External BGP
The command 'execute router clear bgp' is used to restart a BGP session between two peers. It is also used to execute a BGP soft reset, which forces both peers to interchange their complete BGP routing tables.
FortiGate III Student Guide
453
DO NOT REPRINT © FORTINET
External BGP
Follow these steps if you are troubleshooting a BGP problem between two peers: - Check that the local router can reach the remote peer - If the remote peer is not on a connected network, enable EBGP multihop - Check the TCP session - Check the BGP session - If the BGP session is established, check the prefixes received and advertised by each peer.
FortiGate III Student Guide
454
DO NOT REPRINT © FORTINET
External BGP
These are the topics covered in this lesson.
FortiGate III Student Guide
455
DO NOT REPRINT © FORTINET
OSPF
This lesson is about OSPF configuration and troubleshooting.
FortiGate III Student Guide
456
DO NOT REPRINT © FORTINET
OSPF
This lesson reviews some OSPF concepts, such as link state advertisements, cost, adjacencies, and areas. The lesson also covers OSPF troubleshooting and basic OSPF configuration. Similarly to BGP, OSPF is a deep topic more extensive than what is covered in this lesson. This lesson shows only the basics about the protocol and the FortiGate to troubleshoot of the most common OSPF issues.
FortiGate III Student Guide
457
DO NOT REPRINT © FORTINET
OSPF
Let's start with a review of basic OSPF concepts.
FortiGate III Student Guide
458
DO NOT REPRINT © FORTINET
OSPF
In a link state protocol, such as OSPF, every router has a complete view of the network topology. Some of the advantages of OSPF are scalability and fast convergence. Every 30 minutes routers re-advertise their OSPF information. Between those 30-minute intervals, only updates are sent if a topology change is detected. So, it is a relatively quiet protocol as long as the network topology is stable. OSPF in large networks, although, may require good planning and be difficult to troubleshoot.
FortiGate III Student Guide
459
DO NOT REPRINT © FORTINET
OSPF
(slide contains animation) Each router (in the same area) has identical and synchronized databases. We will talk about OSPF areas soon. (click) An OSPF router uses the information in the LSDB and the Dijkstra's algorithm to generate an OSPF tree, which contains the shortest path from the local router to each other router and network. (click) The tree gives the best route to each destination, (click) which is the information that OSPF can inject into the device's routing table.
FortiGate III Student Guide
460
DO NOT REPRINT © FORTINET
OSPF
The topology information interchanged by OSPF peers is contained in link state advertisements (LSAs). A router's LSBD is populated with the information from the local LSAs and all the LSAs received from other routers. A router sends link state update packets, which contain one or more LSAs.
FortiGate III Student Guide
461
DO NOT REPRINT © FORTINET
OSPF
If there are multiple OSPF routes to the same destination subnet, OSPF selects the route with the lowest cost. Each router interface is associated with an interface cost, which is usually related with how fast or preferable that interface is. An OSPF route cost is the sum of all interfaces' costs to the final destination.
FortiGate III Student Guide
462
DO NOT REPRINT © FORTINET
OSPF
The following two slides explain briefly how an OSPF router builds its OSPF tree. The initial information for each router is the locally connected networks, together with the OSPF cost for each interface. For example, the router R2 has three locally connected subnets: subnet Net 1 with a cost of 2, subnet Net 2 with a cost of 3, and subnet Net 3 with a cost of 1. Router R1 has only one subnet connected: Net 1 with a cost of 2, and so on. Each router starts advertising its locally connected subnets by sending LSAs.
FortiGate III Student Guide
463
DO NOT REPRINT © FORTINET
OSPF
Then, OSPF routers use the Dijkstra’s algorithm to determine the best route to each destination. The best routes can be represented as a tree with the local router at the root. The Dijkstra’s algorithm is a recursive process that is repeated multiple times until the best routes are found. For example, this slide shows the OSPF tree for router R2. It indicates that the best route to Net 5 and Net 4 is through R3; and that Net 1, Net 2 and Net 3 are locally connected.
FortiGate III Student Guide
464
DO NOT REPRINT © FORTINET
OSPF
An OSPF network can be segmented into areas. Each area is identified by a unique number, which can be represented either in decimal or IP address format.
FortiGate III Student Guide
465
DO NOT REPRINT © FORTINET
OSPF
Each area has its own separated LSDB. All routers in the same area maintain an identical copy of the area's LSDB. As we will see later, a router can belong to more than one area. In those cases, the router maintains multiple LSDBs, one LSDB for each area connected to it. Segmenting big OSPF networks into areas reduces the sizes of the LSDB tables. Additionally, a topologies change does not impact the whole network, but only the area where the change happens. Using OSPF areas, although, requires good planning and may complicate the troubleshooting process.
FortiGate III Student Guide
466
DO NOT REPRINT © FORTINET
OSPF
Any OSPF network must have at least one area, the backbone area. The backbone is the core of the network with all the other areas connected to it in a hub-and-spoke topology.
FortiGate III Student Guide
467
DO NOT REPRINT © FORTINET
OSPF
An internal OSPF router has all its interfaces connected to the same area. So, it maintains only one LSDB. On the other hand, an area border router is connected to multiple areas, so it keeps multiple LSDBs. A backbone router has at least one interface connected to the backbone area. An autonomous system boundary router (ASBR) redistributes non-OSPF routes into the OSPF network.
FortiGate III Student Guide
468
DO NOT REPRINT © FORTINET
OSPF
This is example that shows each router type.
FortiGate III Student Guide
469
DO NOT REPRINT © FORTINET
OSPF
An OSPF session between two OSPF peers is called an adjacency. This slide shows the initial interchange between two peers that are forming an adjacency. Any new adjacency goes through different states: Init, 2-way, ExStart, Exchange, Loading and Full. The Full state indicates that the adjacency has successfully formed, and both routers have identical copies of the LSDB.
FortiGate III Student Guide
470
DO NOT REPRINT © FORTINET
OSPF
There are the requirements for two peers to form an OSPF adjacency. If any of the requirements is not met, the adjacency fails and will not reach the Full state.
FortiGate III Student Guide
471
DO NOT REPRINT © FORTINET
OSPF
There are two types of OSPF networks. Point-to-point networks contain only two peers, one at each end of a point-to-point link. Multi-access networks support two or more peers. There are two sub-types of multi-access networks. Broadcast networks support multicasting. An example of a broadcast multi-access network is an Ethernet network. Non-broadcast multi-access networks (NBMA) do not support multicasting. An example of a NBMA network is Frame Relay.
FortiGate III Student Guide
472
DO NOT REPRINT © FORTINET
OSPF
In any multi-access network there are one designated router (DR) and one backup designated router (BDR). The router with the highest priority is elected as the DR. If two or more routers are tied with the highest priority, the router with the highest OSPF ID is elected. The BDR monitors the DR status. If the DR fails, the BDR takes the DR role. Other routers form adjacencies only with the DR and the BDR. The DR forwards the link state information from one router to another. This simplifies the amount of adjacencies required in multi-access networks.
FortiGate III Student Guide
473
DO NOT REPRINT © FORTINET
OSPF
These are the multicast addresses used by OSPF in broadcast multi-access, and point-to-point networks. Keep in mind that OSPF also uses unicast addresses, for LSA retransmissions and database description packets.
FortiGate III Student Guide
474
DO NOT REPRINT © FORTINET
OSPF
There are 11 LSA types. This lesson covers the five more commonly used: -Type 1 describes all the links connected to a router -Type 2 describes all the routers (if more than one) in a multi-access network -Type 3 describes the networks within an area (only generated by a ABR) -Type 4 describes the path to reach an ASBR -Type 5 describes the external destinations originated by an ASBR We will see examples of each of these five types in the next slides.
FortiGate III Student Guide
475
DO NOT REPRINT © FORTINET
OSPF
Type 1 describes the networks connected to a router. They are advertised by all the routers in an area. Type-1 LSAs are not advertised outside the area where they originate.
FortiGate III Student Guide
476
DO NOT REPRINT © FORTINET
OSPF
Type-2 LSAs are advertised only by DRs. In this example, the area has two multi-access networks, each of them with one DR. The two DRs advertise type-2 LSAs, which contain information about the other routers connected to their multi-access networks.
FortiGate III Student Guide
477
DO NOT REPRINT © FORTINET
OSPF
Type-3 LSAs contain summarized link state information. They are advertised only by area border routers (ABRs). In this example, the ABR in the left sends type-3 LSAs to the area 1. They contain link state information for the summarized subnets in areas 0 and 2. This same ABR also sends type-3 LSAs to the backbone area with a summary of the subnets in area 1. Something similar happens with the ABR in the right. It sends type-3 LSAs to the area 2. They contain links state information for the summarized subnets in areas 0 and 1. This same ABR also sends type-3 LSAs to the backbone area with a summary of the subnets in area 2.
FortiGate III Student Guide
478
DO NOT REPRINT © FORTINET
OSPF
An ASBR advertises itself by sending type-1 LSAs. These LSAs have the E-bit on in the OSPF header. As any other type 1, the LSAs with the E-bit are confined to the area where they originate. However, ABRs in the same area send a type-4 LSA to the other areas with information about how to reach the ASBR. In this example, an ASBR that is redistributing RIP routes into OSPF announces itself by sending type-1 LSAs to the backbone area. The ABR receives that LSA and sends a type-4 LSA to the area 1.
FortiGate III Student Guide
479
DO NOT REPRINT © FORTINET
OSPF
The last type of LSA covered in this lesson is type 5. They are sent only by the ASBRs and are not confined to one area. Indeed, they reach all the standard areas. They contain link state information for routes redistributed to OSPF (also called external routes). Note: All the area examples in this lesson are standard areas. There are also stub and NSSA areas, which are not covered in this lesson. Type-5 LSAs are not advertised to stub or NSAA areas.
FortiGate III Student Guide
480
DO NOT REPRINT © FORTINET
OSPF
Each external route is assigned a metric. There are two types of external-route metrics. A type-1 metric is the sum of the external cost plus the internal cost to reach the ASBR. A type-2 metric is only the external cost (the internal cost is not considered). If there are two external routes to the same destination, one type 1 and one type 2, an OSPF router selects the type 1 over the type 2.
FortiGate III Student Guide
481
DO NOT REPRINT © FORTINET
OSPF
This is a basic FortiGate OSPF configuration. It has the list of areas, the list of OSPF networks, and the OSPF router ID.
FortiGate III Student Guide
482
DO NOT REPRINT © FORTINET
OSPF
Any FortiGate that is redistributing non-OSPF routes into OSPF is an ASBR.
FortiGate III Student Guide
483
DO NOT REPRINT © FORTINET
OSPF
The next part of this lesson is about OSPF troubleshooting.
FortiGate III Student Guide
484
DO NOT REPRINT © FORTINET
OSPF
'get router info ospf status' shows general OSPF information and some statistics about OSPF.
FortiGate III Student Guide
485
DO NOT REPRINT © FORTINET
OSPF
The output of 'get router info ospf status' also shows information about each area the router belongs to.
FortiGate III Student Guide
486
DO NOT REPRINT © FORTINET
OSPF
(slide contains animation) For OSPF information about each interface, use the command 'get router info ospf interface'. It shows: -Network type, in this case broadcast multi-access (click) - If it is a DR, or a BDR (click) - DR and BDR IP addresses (click) - Number of adjacencies and traffic statistics
FortiGate III Student Guide
487
DO NOT REPRINT © FORTINET
OSPF
This command shows a summary of the status of all the OSPF neighbors. For each neighbor, it displays the adjacency state; and if it is a DR, a BDR, or none of them (DROther). A dash is displayed after the state if the neighbor is in a point-to-point network.
FortiGate III Student Guide
488
DO NOT REPRINT © FORTINET
OSPF
The command 'get router info ospf database brief' provides a summary of all the LSDB entries in the FortiGate, order by LSA types. It shows first the LSAs type 1 (router link states), then type 2 (net link states).
FortiGate III Student Guide
489
DO NOT REPRINT © FORTINET
OSPF
After that, the command shows the LSAs type 3 (summary link state), type 4 (ASBR-summary link states) and type 5 (AS external link states).
FortiGate III Student Guide
490
DO NOT REPRINT © FORTINET
OSPF
The command 'get router info ospf database self-originate ' lists the LSAs originated in the local FortiGate.
FortiGate III Student Guide
491
DO NOT REPRINT © FORTINET
OSPF
To get the details about each LSA, use the command 'get router info ospf database router lsa'.
FortiGate III Student Guide
492
DO NOT REPRINT © FORTINET
OSPF
This is a sample of more output from the command 'get router info ospf database router lsa'.
FortiGate III Student Guide
493
DO NOT REPRINT © FORTINET
OSPF
The OSPF real time debug displays information about adjacency establishments and OSPF errors. It also shows information about network topology changes.
FortiGate III Student Guide
494
DO NOT REPRINT © FORTINET
OSPF
These are the topics covered in this lesson.
FortiGate III Student Guide
495
DO NOT REPRINT © FORTINET
High Availability
This lesson is about HA troubleshooting.
FortiGate III Student Guide
496
DO NOT REPRINT © FORTINET
High Availability
The lesson reviews how virtual MAC addresses are assigned in a HA cluster. It also covers HA monitoring and troubleshooting.
FortiGate III Student Guide
497
DO NOT REPRINT © FORTINET
High Availability
We will start with a quick review of the HA operation material covered in NSE4. There are two HA modes: active-passive and active-active. Let’s examine first the active-passive mode. In any of the two HA operation modes, the configurations of the secondary FortiGates are synchronized with the configuration in the primary device. In the case of the active-passive mode, the primary FortiGate is the only device that actively processes traffic. The secondary FortiGates remain in passive mode monitoring the status of the primary unit. If a problem is detected in the primary FortiGate, one of the secondary devices will take over the primary role. This event is what we call HA failover. Session synchronization is an optional setting (which is disabled by default) that enables seamless failover for some traffic. The information of some sessions is synchronized, so when the primary fails, the new primary can take over those sessions where they were left and keep them open. Traffic might be interrupted for a few seconds, but the network applications will not need to reconnect the sessions again.
FortiGate III Student Guide
498
DO NOT REPRINT © FORTINET
High Availability
Like with active-passive HA, in active-active, all FortiGates’ configurations are synchronized. Also, if a problem is detected with the primary device, one of the secondary units will take over the primary role. However, one of the main differences with active-passive is that in active-active all of the FortiGates are processing traffic. Indeed, one of the tasks of a primary FortiGate in active-active is to balance some of the sessions among all the secondary devices.
FortiGate III Student Guide
499
DO NOT REPRINT © FORTINET
High Availability
There are three reasons why a failover can happen: 1- When the primary stops replying to the heartbeats 2- When the link status of a monitored interface goes down - you can configure a HA cluster to monitor the link status of one or more interfaces 3- When a server (IP address) stops replying to the ping sent by the primary - you can configure a HA cluster to periodically send a ping to one or more servers to test the connectivity between the primary unit and the network services
FortiGate III Student Guide
500
DO NOT REPRINT © FORTINET
High Availability
To forward traffic correctly, a FortiGate HA solution uses virtual MAC addresses. When a primary joins an HA cluster, each interface is given a virtual MAC. The primary informs all secondary units about the assigned virtual MACs. Upon failover, a secondary adopts the same virtual MAC addresses for equivalent interfaces.
FortiGate III Student Guide
501
DO NOT REPRINT © FORTINET
High Availability
After fail over, the new primary also broadcasts gratuitous ARP packets, notifying the network that each virtual MAC address is now reachable through a different switch port. In most of the networks that is enough for the switches to update their MAC forwarding tables with the new information. However, some high-end switches might not clear properly their MAC tables after a fail-over. So, they keep sending packets to the former primary even after receiving the gratuitous ARPs. In those cases, enable the command 'link-failed-signal' to force the former primary to shut down all its nonheartbeat interfaces for 1 second when the failover happens. This simulates a link failure that clears the related entries from the switches' MAC tables.
FortiGate III Student Guide
502
DO NOT REPRINT © FORTINET
High Availability
The HA virtual MAC addresses assigned to each interface are determined by the HA group ID, the virtual cluster ID and the interface index. So, if you have two or more HA clusters in the same broadcast domain, and using the same HA group ID, you might get MAC address conflicts. For those cases, it is strongly recommended to assign different HA group IDs to each cluster.
FortiGate III Student Guide
503
DO NOT REPRINT © FORTINET
High Availability
The same command we used to get interface traffic statistics can be used to display the HA virtual MAC address assigned to an interface.
FortiGate III Student Guide
504
DO NOT REPRINT © FORTINET
High Availability
(slide contains animation) Let’s show how a HA cluster in active-active mode distributes traffic. (click) First, the client side sends a SYN packet. It’s forwarded always to the primary FortiGate using the internal interface’s virtual MAC address as the destination. (click) If the primary decides that the session is going to be inspected by a secondary, the primary forwards the SYN packet to the respective secondary. In this case, the destination MAC address is the physical MAC address of the secondary FortiGate. (click) The secondary responds with SYN/ACK to the client and starts the connection with the server by directly sending a SYN packet.
FortiGate III Student Guide
505
DO NOT REPRINT © FORTINET
High Availability
(slide contains animation) Next the client acknowledges the SYN/ACK. It’s forwarded again to the primary using the virtual MAC address as the destination. (click) The primary device forwards the packet to the secondary inspecting that session, using the secondary’s physical MAC address.
FortiGate III Student Guide
506
DO NOT REPRINT © FORTINET
High Availability
(slide contains animation) When the server responds to the TCP SYN, again, the packet is sent to the primary using the external interface’s virtual MAC. (click) So the primary signals the secondary. (click) And it is the secondary the one who replies to the server. As you can see, the idea of active-active mode is not to load balance bandwidth. The traffic is always sent first to the primary. The main objective is to share CPU and memory among multiple FortiGates for traffic inspection.
FortiGate III Student Guide
507
DO NOT REPRINT © FORTINET
High Availability
If you connect to a secondary's console port while it is joining a HA cluster, these are the messages you should see. The secondary tries to synchronize first what are called the "external files". They include the FortiGuard databases and digital certificates. After that, the secondary synchronizes the configuration. The last message indicates that the secondary has successfully joined the cluster.
FortiGate III Student Guide
508
DO NOT REPRINT © FORTINET
High Availability
If the HA cluster has formed successfully, the GUI displays all the FortiGates, together with their hostnames and serial numbers.
FortiGate III Student Guide
509
DO NOT REPRINT © FORTINET
High Availability
When troubleshooting any problem in a HA cluster, it is useful to know that you can connect to the CLI of any secondary from the primary CLI. You have to use the command 'execute ha manage' with the secondary HA index for that purpose. To get the list of secondary FortiGates with their HA indexes, use the question mark at the end of that same command.
FortiGate III Student Guide
510
DO NOT REPRINT © FORTINET
High Availability
(slide contains animations) From the CLI, although, you can get more information about the status of the HA. For example, the command 'diagnose sys ha status' displays heartbeat traffic statistics, (click) as well as the serial number and HA priority of each FortiGate. The command also shows the heartbeat interface IP address automatically assigned to the primary FortiGate.
FortiGate III Student Guide
511
DO NOT REPRINT © FORTINET
High Availability
As explained in NSE4, the HA uptime is one of variables used to elect the primary unit. Depending on other variables and configuration, at one point the units might compare their system uptimes to elect the primary. If that happens and if there is one unit whose system uptime is 5 minutes more than the system uptimes of all the other units, it is elected primary. This command can be used to compare the system uptimes of all the units in a cluster.
FortiGate III Student Guide
512
DO NOT REPRINT © FORTINET
High Availability
A good indication of the health of a HA cluster is the status of the configuration synchronization. To check that all the secondary configurations are synchronized with the primary configuration, you can execute the command 'diagnose sys ha showcsum' in all the HA units. If a secondary FortiGate displays exactly the same sequence of numbers than the primary, its configuration it's well synchronized. Also, in each of the devices, the debugzone and the checksum zone must display the same sequence of numbers. We will suggest later some troubleshooting tips if that is not case.
FortiGate III Student Guide
513
DO NOT REPRINT © FORTINET
High Availability
Instead of running the previous command in each of the cluster units, you can alternatively run this other command only on the primary. It shows the checksum for all the cluster members. This command is easier to use. However, if there are communication problems between one of the secondary units and the primary, you might need to use the previous command instead.
FortiGate III Student Guide
514
DO NOT REPRINT © FORTINET
High Availability
FortiGate HA uses FGCP, the FortiGate clustering protocol, for HA-related communications. FGCP travels among the clustered FortiGate units over the links that you have designated as the heartbeats. The FGCP traffic uses a different Ethernet type than the IP protocol. It actually uses three different Ethernet types depending on the operation mode (transparent or NAT/route).
FortiGate III Student Guide
515
DO NOT REPRINT © FORTINET
High Availability
HA session synchronization, as explained, is disabled by default. If it is enabled, you can check in the primary's session table which sessions have been synchronized to the secondary units. They are the ones with the synced flag. Additionally and for the case of all sessions, the ha_id field shows the HA member ID of the unit that is processing the traffic.
FortiGate III Student Guide
516
DO NOT REPRINT © FORTINET
High Availability
If a failover happens, the best tool to get information about it is the FortiGate logs. In case the failover happened because the primary unit failed, the secondary unit's logs should show these log entries.
FortiGate III Student Guide
517
DO NOT REPRINT © FORTINET
High Availability
If a new primary was elected because one or more monitored interfaces failed, the former primary displays logs similar to these ones. In this case the primary unit is reporting a problem with the monitored interface port1.
FortiGate III Student Guide
518
DO NOT REPRINT © FORTINET
High Availability
If a unit cannot join a cluster follow these steps: - Verify the HA settings - Verify the firmware versions and hardware models - Verify the physical layer connections - Use the HA real time debug while the unit tries to join the cluster. Run the debug both in the primary and the unit with the problem If the checksums between the debugzone and checksum zones do not match, you can force the recalculation.
FortiGate III Student Guide
519
DO NOT REPRINT © FORTINET
High Availability
Traffic from session synchronization is bandwidth intensive. If the session creation rate is high, session synchronization traffic might interfere with the heartbeat traffic, creating delays in the heartbeat replies. There two configuration changes that might help in those cases: 1- Use an interface different than the heartbeat interface for session synchronization 2- Delay the synchronization of new sessions by 30 seconds, so short-lived sessions are not synchronized High CPU issues could also create HA heartbeat problems. In those cases, troubleshoot and fix the high CPU problem first before checking the HA status.
FortiGate III Student Guide
520
DO NOT REPRINT © FORTINET
High Availability
These are the topics covered in this lesson.
FortiGate III Student Guide
521
View more...
Comments
