VSX

CoreXL and CPU affinity

[Restricted] ONLY for designated groups and individuals 2012 Check Point Software Technologies Ltd.
Course Timetables

Day 1 Day 2 Day 3

9:00 Course Introduction VSX Clustering VSX Conversion

10:00 RC & QoS

vsx_utill
R75.40VS VSX Gaia VS CTX & New Features
11:00
Introduction (Conversion, SNMP, JF)
12:00 Mgmt. Implementation L2 VS

13:00 Lunch Break

14:00
VSX Networking GW Implementation
15:00
Meeting with Check Point
R&D
16:00
VSX CoreXL Affinity &
Debug & Troubleshooting
Memory RC
17:00

[Restricted] ONLY for designated groups and individuals 2012 Check Point Software Technologies Ltd. 2
CoreXL

CoreXL architecture
Parallelise security gateway kernel
Leverage modern processor architectures
Suited to medium path

[Restricted] ONLY for designated groups and individuals 2012 Check Point Software Technologies Ltd. 3
Security Gateway CoreXL
Firewall kernel Replication
Firewall kernel is replicated multiple times. Each runs on one
processing core.
Each instance is independent FW-1 kernel.
Instances can run concurrently dont share a global lock.

Dispatcher
New component introduced in CoreXL.
Receives packets and forwards them to the kernel instances.
Acts as a load balancer. The dispatching is based on a hash of the
source IP, Destination IP, Destination port and IP protocol (4-tuple)
The dispatcher must maintain core stickiness per connection

[Restricted] ONLY for designated groups and individuals 2012 Check Point Software Technologies Ltd. 4
CoreXL - First Packet Flow

IP stack
Record
Conn

fw0 conn
table
fw1 conn
table
fw2 conn
table
WT WT WT
2
Queue Queue Queue

Arbitrary
Decision
Lookup. Dispatcher global conn table

Not found

PKT
[Restricted] ONLY for designated groups and individuals 2012 Check Point Software Technologies Ltd. 5
CoreXL - Second Packet Flow

IP stack

fw0 conn
table
fw1 conn
table
fw2 conn
table
WT WT WT

Queue Queue Queue

Lookup. Dispatcher global conn table

Found 2

PKT
[Restricted] ONLY for designated groups and individuals 2012 Check Point Software Technologies Ltd. 6
CoreXL - Parallel Processing

IP stack

fw0 conn
table
fw1 conn
table
fw2 conn
table
WT WT WT

Queue Queue Queue

Dispatcher global conn table

0 1 2

PKT
[Restricted] ONLY for designated groups and individuals 2012 Check Point Software Technologies Ltd. 7
 CoreXL

Core #0 Core #1 Core #2 Core #3

eth0

eth1
Dispatcher Dispatcher
fw5 fw4
SND

SND
Medium Path Medium Path
PPAK PPAK
Queue Queue

Core #4 Core #5 Core #6 Core #7

fw3
Medium Path
fw2Medium Path
fw1
Medium Path
fw0Medium Path
Queue Queue Queue Queue

Accelerated Path Cores are allocated via Interface IRQ

Affinity
Secure Network Dispatcher queues packets to firewall
instances running Firewall and Medium Paths
[Restricted] ONLY for designated groups and individuals 2012 Check Point Software Technologies Ltd. 8
 Accelerated Path No Template

Core #... Core #4 Core #... C

FW Medium FW Medium FW Medium

Path Path Path Path Path Path
Queue Queue Queue

Core #0 Core #1
Dispatcher Dispatcher
Performance Pack Performance Pack

eth0 eth1

Syn
SynAck + subsequent S2C packets
Subsequent C2S packets
[Restricted] ONLY for designated groups and individuals 2012 Check Point Software Technologies Ltd. 9
 Accelerated Path With Template

Core #... Core #4 Core #... C

FW Medium FW Medium FW Medium

Path Path Path Path Path Path
Queue Queue Queue

Core #0 Core #1
Dispatcher Dispatcher
Performance Pack Performance Pack

eth0 eth1

Syn + subsequent C2S packets

SynAck + subsequent S2C packets

[Restricted] ONLY for designated groups and individuals 2012 Check Point Software Technologies Ltd. 10
 Medium Path IPS Traffic

Core #... Core #4 Core #... C

FW Medium FW Medium FW Medium

Path Path Path Path Path Path
Queue Queue Queue

Core #0 Core #1
Secure Dispatcher Secure Dispatcher
Performance Pack Performance Pack

eth0 eth1

Syn + subsequent C2S packets

SynAck + subsequent S2C packets

[Restricted] ONLY for designated groups and individuals 2012 Check Point Software Technologies Ltd. 11
VSX CoreXL

VSX CoreXL
Same idea as applied for SG is applied to VSX CoreXL.
Main difference, instance in FWK (fw kernel equivalent) are
executed by UM threads.
VSX CoreXL can be applied for any existing VS
simultaneously with different number of instances.

[Restricted] ONLY for designated groups and individuals 2012 Check Point Software Technologies Ltd. 12
VSX CoreXL cont.

VSX CoreXL affinity

VSX CoreXL does not affine FWK instance per core.
Affinity can be set manually per instance.

VSX Semi Static affinity

Semi Static affinity will assign FWK to run on a default number of cores.
The number of cores is calculated using a formula.
This number can also be changed by a manual command (fwkall).
The cores chosen will have a physical proximity.
Any manual settings to FWK or FWK instance will override the semi static
affinity.

[Restricted] ONLY for designated groups and individuals 2012 Check Point Software Technologies Ltd. 13
VSX CoreXL configuration
CoreXL configuration for VS0 is done using cpconfig
This program will let you re-configure
your Check Point products configuration.

Configuration Options:
----------------------
(1) Licenses and contracts
(2) SNMP Extension
(3) PKCS#11 Token
(4) Random Pool
(5) Secure Internal Communication
(6) Enable cluster membership for this gateway
(7) Disable Check Point SecureXL
(8) Configure Check Point CoreXL
(9) Automatic start of Check Point Products

(10) Exit

Enter your choice (1-10) :

CoreXL for VS which is not VS0 is done using SmartDashboard

Note: changing CoreXL configuration (num of instances) will require downtime of the VS (VS0 or other).

[Restricted] ONLY for designated groups and individuals 2012 Check Point Software Technologies Ltd. 14
VSX Affinity

VSX affinity handles the following VSX entities

VS - Setting affinity for VS means setting affinity for all
processes related to the specific VS (all processes with the
specific vrf)

Process - Setting affinity is possible for specific process per

specific VS.

FWK instance Setting affinity for FWK instance means setting

affinity per FWK thread (any FWK instance for any VS)

FWKALL Setting affinity for all FWK processes by the number

of CPUs, the specific cores are chosen by the gateway.

[Restricted] ONLY for designated groups and individuals 2012 Check Point Software Technologies Ltd. 15
VSX Affinity

Affinity persistency - Vsx affinity is maintained over reboot and over

process crash cycle using configuration files

Affinity Exceptions VSX affinity can handle process exceptions

which will be chosen by the user. VSX affinity does not affine kernel
threads.

Affinity Priority VSX affinity entities can be set together in the same
VS, using the following priority
FWK instance
Process
VS
In case of override the user will be prompt for actions

[Restricted] ONLY for designated groups and individuals 2012 Check Point Software Technologies Ltd. 16
VSX Affinity Usage
Setting Affinity
Interface Affinity: fw ctl affinity -s -i <interface> <cpuids | all>
VS affinity (VS,VR,VSW): fw ctl affinity -s -d [-vsid <ranges>] -cpu <ranges>
Process affinity - fw ctl affinity -s -d -pname <process name> [-vsid <ranges>] -cpu <ranges>
pid Affinity - fw ctl affinity -s -p <pid> <cpuids | all>
FWK instance affinity - fw ctl affinity -s -d -inst <ranges> -cpu <ranges>
All FWKs affinity - fw ctl affinity -s -d -fwkall <num of CPUs>
Note: If vsid flag is omitted, the current context will be used.

Listing Affinity
Configured affinity - fw ctl affinity -l
Extended Affinity - fw ctl affinity -l -x [-vsid <ranges>] [-cpu <ranges>] [-flags e|k|t|n]
Flags:
e don't print exception processes
k don't print kernel threads
t print also all process threads
n print process name instead of /proc/<pid>/cmdline
h print CPU mask in hex format

[Restricted] ONLY for designated groups and individuals 2012 Check Point Software Technologies Ltd. 17
Usage Examples
Setting affinity examples
fw ctl affinity -s -d -fwkall 3
fw ctl affinity -i eth0 0 3 7
fw ctl affinity -s -d -inst 0 2 4 -cpu 5
fw ctl affinity -s -d -pname cpd -vsid 0-12 -cpu 7
fw ctl affinity -s -d -vsid 0-2 4 6-8 -cpu 0-2 4

Listing Affinity example

fw ctl affinity -l
Output:
eth0: CPU 1
VS_0 FWK_INSTANCE_0: CPU 0 1 2
VS_0 fwk: CPU 2 3
VS_1 FWK_INSTANCE_0: CPU 0
VS_1 FWK_INSTANCE_1: CPU 1
VS_1 FWK_INSTANCE_2: CPU 2
VS_1 fwk: CPU 2 3
VS_2 cpd: CPU 1 2 3
VS_2 fwk: CPU 2 3
VS_3 fwd: CPU 1 3
VS_3 fwk: CPU 0 3

[Restricted] ONLY for designated groups and individuals 2012 Check Point Software Technologies Ltd. 18
Usage Examples (cont)
Extended Affinity List example
fw ctl affinity l x vsid 1 flags tnek
Output:
-------------------------------------------------------
|PID |VSID | CPU |SRC|V|KT |EXC| NAME
-------------------------------------------------------
| 4835 | 1 | all | | | | | routed
| 21094 | 1 | all | | | | | fwk_wd
| 21096 | 1 | all | | | | | cpd
| 21241 | 1 | all | | | | | |---cpd
| 21244 | 1 | all | | | | | |---cpd
| 21245 | 1 | all | | | | | |---cpd
| 21107 | 1 | all | | | | | mpdaemon
| 21115 | 1 | 2 3 | P | | | | fwk1_dev
| 21116 | 1 | 0 | I | | | | |---fwk1_0
| 21117 | 1 | 1 | I | | | | |---fwk1_1
| 21118 | 1 | 2 | I | | | | |---fwk1_2
| 21119 | 1 | 2 3 | P | | | | |---fwk1_3
| 21401 | 1 | all | | | | | fw
| 21411 | 1 | all | | | | | |---fw
| 21412 | 1 | all | | | | | |---fw
| 21413 | 1 | all | | | | | |---fw
-------------------------------------------------------

[Restricted] ONLY for designated groups and individuals 2012 Check Point Software Technologies Ltd. 19
CoreXL with Affinity Example

Command used for viewing fwk setup in the following example

fw ctl affinity l x vsid 1 flags tn | grep fwk | grep v fwk_

Before Setting CoreXL

| 21115 | 1 | 2 3 | P | | | | fwk1_dev
| 21116 | 1 | 0 | I | | | | |---fwk1_0

After Setting CoreXL (Used SDB to configure 4 instances)

| 21115 | 1 | 2 3 | P | | | | fwk1_dev
| 21116 | 1 | 2 3 | P | | | | |---fwk1_0
| 21117 | 1 | 2 3 | P | | | | |---fwk1_1
| 21118 | 1 | 2 3 | P | | | | |---fwk1_2
| 21119 | 1 | 2 3 | P | | | | |---fwk1_3

Set affinity for instance 0 (fw ctl affinity s d inst 0 cpu 0)

| 21115 | 1 | 2 3 | P | | | | fwk1_dev
| 21116 | 1 | 0 | I | | | | |---fwk1_0
| 21117 | 1 | 2 3 | P | | | | |---fwk1_1
| 21118 | 1 | 2 3 | P | | | | |---fwk1_2
| 21119 | 1 | 2 3 | P | | | | |---fwk1_3

Set affinity for instance 2 and 3 (fw ctl affinity s d inst 2 3 cpu 1 2)
| 21115 | 1 | 2 3 | P | | | | fwk1_dev
| 21116 | 1 | 0 | I | | | | |---fwk1_0
| 21117 | 1 | 2 3 | P | | | | |---fwk1_1
| 21118 | 1 | 1 2 | I | | | | |---fwk1_2
| 21119 | 1 | 1 2 | I | | | | |---fwk1_3

[Restricted] ONLY for designated groups and individuals 2012 Check Point Software Technologies Ltd. 20