3.9 Clustering
PacketiX VPN Server 2.0 Enterprise Edition & PacketiX VPN Server 2.0
Carrier Edition both support the clustering function. This section
explains the clustering function.
3.9.1 What is Clustering?
Clustering Necessity
In general terms, clustering is a processing method which enables the
distribution of a large processing load which would be difficult for a
single computer to handle among multiple computers, and which appears to
the user as a single system such that there is no need to be aware of
the fact that it is coordinated processing performed in the background
by multiple computers.
PacketiX VPN Server 2.0 Enterprise Edition & Carrier Edition feature
the clustering function, which gathers the VPN Server computers into one
cluster to enable the handling of a large amount of processing as a
whole where a single computer would normally not be capable.
Aims of the PacketiX VPN Server Clustering Function
The PacketiX VPN Server clustering function is designed and
implemented to create the following two types of networks or a single
network combining both. It is not designed or implemented for any other
purposes (for example, for separating a cluster node to a remote
location and running said mode autonomously etc.).
Clustering Function Overview
When configuring a cluster with multiple VPN Server computers, one of
the computers is run in cluster controller mode, while the other
computers are cluster connected to the cluster controller and operated
as cluster member servers. The VPN Server operates in stand alone mode
by default after installation and no clusters are configured.
Clustering enables the following.
- In environments where a large amount of VPN connections need to
be processed, clustering enables their skilful integrated processing
on multiple VPN Servers through load sharing where one computer
attempting the same task would be incapable or have a serious impact
on performance.
-
Where a cluster member server running within the cluster stops
temporarily due to a hardware problem or software update, the
processing to be carried out by that server is automatically taken
over by another cluster member server. Therefore, while long-term
operation of individual servers may result in a malfunction, as a
whole the servers can continue to run almost without disruption.
- When operating a Virtual HUB within the cluster, it is possible
to select either a static Virtual HUB or dynamic Virtual HUB as the
operating mode depending on the objective.
- The entire VPN Server Administrator and the Virtual HUB
Administrators can connect only to the cluster controller and
perform cluster member server administration simply without having
to be aware of their individual existence.
Prerequisites
A VPN Server Enterprise Edition License or Carrier Edition License is
required for each VPN Server when creating a cluster between multiple
VPN Servers.
It is also recommended to connect the VPN Servers to a network with
minimal delays and high throughput. Typically when joining a cluster,
each server is set up in the same location. In this case, it is probably
most desirable for all cluster member servers to be directly connected
to the cluster controller on the same segment without traveling via a
router. Although performance declines, it is technically possible to set
up the cluster controller and cluster member servers in separate
locations via a router. In either case, the cluster controller must be
set up in a location which allows TCP/IP protocol communication from all
other cluster member servers.
Fig. 3-9-1 Connecting the cluster controller & cluster
member servers |
3.9.2 Cluster Controllers
What is a Cluster Controller?
The cluster controller is the computer forming the core of the entire
cluster. The computer representing the cluster when it is created is
known as the cluster controller, and a VPN Client of VPN Server /
VPN Bridge attempting to connect to the cluster designates the
cluster controller's IP address or host name as the destination IP
address or host name.
Overview of Cluster Controller Load Sharing
When the cluster controller receives a VPN connection from a VPN
source computer it performs user authentication in the same manner as a
regular VPN connection. After successful user authentication, the
cluster controller decides automatically which cluster member server is
to perform the processing and realizes load sharing by redirecting the
connection to that cluster member server. The VPN Server which is the
cluster controller is itself also a load sharing destination. The load
sharing algorithm compares the load of each VPN Server and automatically
determined the assignment destination of a newly-connected VPN session.
At this time it uses integers referred to as points in the cluster
member list. By presetting the [Function Standard Ratio in Cluster]
settings entry for the cluster controller and cluster member servers, it
is also possible to manually adjust the parameters for load sharing.
The load sharing discussed here is an overview, and more detailed
control is performed depending on the type of Virtual HUB to which the
actual VPN connection is made. For details, please refer to 「3.9.7 Static Virtual HUBs」 and
「3.9.8 Dynamic Virtual HUBs」.
Setting a VPN Server as a Cluster Controller
The VPN Server operates as a stand alone server in the default
operating mode. Changing this operating mode to a cluster controller
allows the VPN Server to run in cluster controller mode. This and all
other settings related to clustering can only be performed by the entire
VPN Server Administrator.
To set the VPN Server to cluster controller mode, click on the
[Clustering Configuration] button in the VPN Server Manager. Next select
[Cluster controller] in the dialog box which appears and click [OK]. In
the vpncmd utility, use the [ClusterSettingController] command.
Fig. 3-9-2 Configure Clustering settings window |
| Using the PacketiX VPN Server in
cluster mode makes some functions unavailable. Of the functions
used in stand alone server mode, please note that the
configuration data relating to functions described in 「3.9.12 Functions not Available Simultaneously with Clustering」
are all deleted when changing the server operating mode to
cluster controller mode or cluster member server mode. It is
therefore recommended to make a back up before changing the
server's operating mode. |
Creating & Administering Virtual HUBs
Virtual HUBs are created for the cluster controller in the VPN Server
clustering environment. Where necessary, the cluster member servers
create temporary Virtual HUB instances upon instructions from the
cluster controller, but it is not necessary to directly create Virtual
HUBs for the cluster member servers. As explained in 「3.9.10 Collectively Administering the Entire Cluster」, Virtual
HUB creation and the setting & administration of all Virtual HUBs can
only be carried out for the cluster controller in a clustering
environment.
3.9.3 Cluster Member Servers
What is a Cluster Member Server?
The term "cluster member server" refers to any computer which forms a
part of the cluster configuration other than the cluster controller. By
cluster connecting to the cluster controller, the cluster member server
is placed under the control of the said cluster controller and shares
the processing within the cluster.
When adding a cluster member server to an existing server, the
cluster controller's host name or IP address, port number (one of the
listener ports made available by the cluster controller) and
administration password are required.
Setting a VPN Server as a Cluster Member Server
The VPN Server operates as a stand alone server in the default
operating mode. Changing this operating mode to a cluster member server
allows the VPN Server to run in cluster member server mode.
To set the VPN Server to cluster member server mode, click on the
[Clustering Configuration] button in the VPN Server Manager. Next select
[Cluster Member Server] in the dialog box which appears and click [OK].
In the vpncmd utility, use the [ClusterSettingMember] command.
The entries required at this time are as follows.
| Entry |
Description |
| Controller Host
Name or IP Address |
Designates the host name or IP address of the cluster
controller computer representing the cluster. The VPN Server
must be operating in cluster controller mode on the host
designated here. |
| Port Number of
Controller |
Designates the TCP/IP port of the destination cluster
controller. |
| Administration
Password |
Designates the administration password of the destination
cluster controller. Participation as a member in the cluster
is either allowed or denied depending on whether the hash
value of the inserted administration password is matched by
challenge-response authentication. It is also necessary to
change the cluster connection settings' administration
password of the cluster member servers when the cluster
controller administration password is changed. This password
is not associated with the VPN Server administration
password of the cluster member server itself. |
| Public IP Address |
The public IP address of this cluster member filed with the
cluster controller. The IP address designated here is used
as the redirect address when this cluster member server is
selected by the cluster controller as a VPN connection
session load sharing destination from a new VPN source. When
no address is entered, the network interface IP address used
when cluster connecting to the cluster controller is
automatically used. If wishing to use a different public IP
address to that of the network interface when cluster
connecting to the cluster controller, that address should be
designated. |
| Public Port List |
The public port number of this cluster member filed with the
cluster controller. Typically, the list of the listener port
made public by the cluster member server is designated. More
than one public port number must to be designated, and
multiple port numbers can be designated by separating them
with a comma. |
Cluster Connecting to a Cluster Controller with a Cluster Member
Mode VPN Server
VPN Servers running in cluster member mode are constantly connected
to the cluster controller by a special control TCP/IP connection known
as a "cluster connection". The cluster member server attempts to
maintain the control cluster connection between the designated cluster
controller as far as possible. In addition, if the cluster connection is
disconnected or fails to connect, ongoing repeated attempts are made at
an interval of a few seconds until the connection is successful.
When seeking to confirm whether the cluster member server is properly
connected to the cluster controller, connect to the cluster member
server with the VPN Server Manager and click [Clustering Status] to
display the following information. In the vpncmd utility, use the
[ClusterConnectionStatusGet] command.
| Entry |
Description |
| Connection Status |
Displays [Online] when the cluster connection is in normal
status. If the cluster connection is not properly connected,
the cause of the error is displayed. |
| Connection Start
Time |
The time & date at which the cluster connection commenced. |
| Time of First
Successful Connection |
Time & date of first successful connection to cluster
controller. |
| Time of Current
Successful Connection |
Time & date of currently-connected cluster connection. |
| Connection Attempts |
Displays the number of attempts to connect to the cluster
controller to date. |
| Successful
Connections |
Displays the number of connection attempts to date which
were successful. |
| Failed Connections |
Displays the number of connection attempts to date which
failed. |
Fig. 3-9-3 Cluster controller connection status display
window |
Obtaining List of VPN Servers connected to Cluster Controller &
Displaying Details
Connect to the cluster controller with the VPN Server Manager and
click the [Clustering Status] button to display a list of all cluster
controllers and cluster member servers connected to that cluster
controller. In the vpncmd utility, use the [ClusterMemberList]
command.
Fig. 3-9-4 Intra-cluster VPN Server list administration
window |
The entries listed here are as follows.
| Entry |
Description |
| Type |
Either [Controller] or [Member]. |
| Connection Time |
Time & date that member started operating as a member of the
cluster after cluster connection to the cluster controller. |
| Host |
Host name of cluster controller or cluster member server. |
| Points |
Value indicating the load status of the cluster member
server. The higher this value, the lower the load and the
higher the likelihood that the member will be designated as
the load share destination for a new VPN session. |
| Sessions |
Displays the number of VPN sessions being processed by the
VPN Server. |
| TCP Connections |
Displays the number of TCP/IP connections being processed by
the VPN Server. |
| Dynamic Virtual
HUBs |
Displays the number of instances of Virtual HUBs operating
on the VPN Server. |
| Consumed Client
Connection Licenses |
Displays the number of client connection licenses for the
cluster consumed by the VPN Server. |
| Consumed Bridge
Connection Licenses |
Displays the number of bridge connection licenses for the
cluster consumed by the VPN Server. |
Note that the information for the cluster controller and each of the
cluster member servers displayed in the table of the [Cluster Member
List] dialog box is not the latest information, but is instead a few
seconds old because it is the result of a query made by the cluster
controller to each member server every few seconds.
Also, selecting the desired cluster member server shown in the VPN
Server Manager and clicking on [Cluster Member Server Information]
enables detailed information on that cluster member server to be viewed.
In the vpncmd utility, use the [ClusterMemberInfoGet] command.
Fig. 3-9-5 Intra-cluster member server status display window |
| While cluster connection
communication between a cluster controller and cluster member
servers is TCP/IP protocol-based, it differs from the PacketiX
VPN protocol in that it is implemented by a proprietary
dedicated synchronous and asynchronous RPC (remote proxy call).
The System Administrator does not require an in-depth knowledge
this protocol. Additionally SSL encryption is used on the
protocol contents and a hashed password is used for
authentication. However, it does not feature functions such as
the sophisticated server certificate authentication of the
PacketiX VPN protocol. It is therefore recommended to perform
the cluster connection between the cluster controller and
cluster members using a physically secure range such as the same
LAN. In most cases, there is no problem because all computers
used in the cluster are set up in the same room but caution is
required where the computers are required to be geographically
separated. |
3.9.4 Load Balancing
When making a normal VPN connection from the VPN Client and a cascade
connection from the VPN Client / VPN Bridge to a cluster, designate the
cluster controller's IP address and port number and the name of the
destination Virtual HUB.
The cluster controller VPN Server receiving the connection from the
VPN source carries out authentication of that connection then selects
the cluster member to which to assign that VPN session. The following
algorithms are used in this case.
When the Virtual HUB Designated as the VPN Destination is Static:
The cluster controller redirects the connection to the VPN Server
with the highest point value among all of those currently available.
Please refer to 「3.9.7 Static Virtual HUBs」 for details on static Virtual HUBs.
When the Virtual HUB Designated as the VPN Destination is Dynamic:
The redirect VPN Server is selected according to the following
procedure.
- When the VPN session connected to that Virtual HUB does not yet
exist on one of the VPN Servers in the cluster, the connection is
redirected to the VPN Server with the highest point value.
- When the VPN session connected to that Virtual HUB already
exists on one of the VPN Servers in the cluster, the connection is
redirected to that VPN Server.
Please refer to 「3.9.8 Dynamic Virtual HUBs」 for details on dynamic Virtual HUBs.
3.9.5 Load Balancing using Performance Standard Ratio
Weighting by Performance Standard Ratio
As previously mentioned, when the cluster controller selects the
server with the lowest load from among the VPN Servers in the cluster,
it selects the VPN Server with the highest point value.
The points used here are approximately determined by the following
formula.
| Points = (4096 - No. of sessions
processed by VPN Server × 100 / weight) × 100000 ÷ 4096 |
* PacketiX VPN Server 2.0 Carrier
Edition substitutes 4096 with 100000.
The above formula enables a definition of the performance standard
ratio of each VPN Server by setting "weighting" parameters for each
server. By setting the values of the [Function Standard Ratio in
Cluster] settings entry in the VPN Server's [Configure Clustering], it
is possible to change the weight parameter freely. The default setting
of the weight parameter is 100.
The [Function Standard Ratio in Cluster] value sets how the subject
VPN Server performs against a value of 100 for a normally performing VPN
Server. For example, where two servers have respective [Function
Standard Ratio in Cluster] values of 100 and 200, this means that the
latter server is capable of processing twice the amount of VPN sessions
as the former server. The VPN cluster controller determines how many VPN
sessions the entire VPN Server should be able to process based largely
on the value set here and distributes load accordingly.
Settings to Prevent the Cluster Controller Itself from Processing
VPN Communication
The cluster controller may select itself as the VPN Server to process
a VPN connection from a VPN source. When the cluster controller decides
the VPN Server to which to allocate a new VPN session, the decision is
based on the cluster's VPN Server point values determined by the
algorithms described in 「3.9.4 Load Balancing」, so both the cluster controller and the
cluster members are judged according to an equal standard.
However, when a large volume of VPN connection sessions representing
a significantly large load for the entire server have to be processed,
it is possible to reduce the load on the cluster controller itself by
having it only assume the role of processing the redirection of VPN
sessions to each of the cluster members. To enable this setting, open
the [Configure Clustering] settings entry and enable the [Controller
functions only (It does not process VPN communication itself)] checkbox.
This prevents the cluster controller from selecting itself when deciding
which VPN Server to assign a new VPN session to.
3.9.6 Fault Tolerance
The PacketiX VPN Server cluster system not only offers load balancing
but also realizes fault tolerance at the same time.
When a cluster member server within the cluster terminates suddenly due
to hardware trouble or a software / device driver malfunction, or when a
situation arises whereby it has to temporarily terminate its VPN Server
process in order to update its VPN Server software program and OS, that
cluster member server loses connection with the cluster controller, such
that the cluster controller automatically deems it as having disengaged
from the cluster and excludes it from the load balancing.
In addition, all VPN session which were connected to the cluster
member server which has ceased to function are automatically taken over
by other cluster member servers. This processing is carried out
automatically without the need for any special handling by the VPN
client computer of the VPN source. Therefore, even when a part of the
multiple VPN Server computers used by an ISP or a large company
terminate due to a malfunction or have to shut down for maintenance,
this mechanism enables the entire network to continue operating without
stopping as long as other computers remain in the cluster.
Fig. 3-9-6 Realizing fault tolerance with the PacketiX VPN
Cluster |
3.9.7 Static Virtual HUBs
Virtual HUBs not using clustering are not particularly classified,
but in a clustering environment they are classified into two types:
static Virtual HUBs and dynamic Virtual HUBs. While the Virtual HUB's
type has to be designated upon creation, it is also possible to change
the type at a later date.
First is an explanation of static hubs.
A static Virtual HUB is used to conveniently create a Virtual HUB
for remote access VPN. Creating a static Virtual HUB within a cluster
generates that hub's instance (entity) in all VPN Servers within the
cluster, which continues to run on all VPN Servers as long as the
cluster is operating.
When connection source VPN software (usually an end user VPN Client)
wishing to make a remote access connection is connected to the cluster
controller, the cluster controller uses the aforementioned algorithms to
select one of the VPN Servers and redirects the connection to the static
Virtual HUB instance within that VPN Server.
By configuring a local bridge connection between the physical Network
Adapters connected to each of the VPN Servers for each static Virtual
HUB instance created in each VPN Server in the cluster, and by
connecting all of the local bridging destination physical LANs to the
in-house LAN destination to which the remote access is desired (either a
direct layer 2 connection or a layer 3 connection using a router and NAT
is acceptable), the VPN Client user can remotely access this in-house
LAN regardless of which VPN Server the connection is assigned to.
This mechanism enables the creation of a large-scale remote access
VPN service required to process a large volume of simultaneous
connections. Please refer to 「10.8 Setting Up a Large Scale Remote Access VPN Service」 for specific configurations.
3.9.8 Dynamic Virtual HUBs
The dynamic Virtual HUB is a type of Virtual HUB convenient for
providing VPN Server services such as creating a large number of Virtual
HUBs within a cluster and enabling users connected to the same Virtual
HUB to communicate freely. Dynamic Virtual HUBs are suitable, for
instance, as a way for systems divisions of large companies to make
Virtual HUBs for each department, or for ISPs creating Virtual HUBs as a
service to their customers, wherein those departments and customers have
the administration authority for that Virtual HUB and are free to
operate it as they please. Such uses only require the entire VPN Server
Administrator to take note of whether the VPN cluster is running
properly, and all of the responsibility for setting and administering
each Virtual HUB can be delegated to the Virtual HUB Administrators.
When a dynamic Virtual HUB has been created within a cluster but does
not have any one connected to it, that Virtual HUB's instance (entity)
does not exist on any of the VPN Servers in the cluster. When the first
session designating that Virtual HUB makes a VPN connection, the
controller selects the VPN Server which should launch that Virtual HUB's
instance for the first time, then creates the Virtual HUB instance for
that VPN Server and redirects the VPN session to that server. For the
second and subsequent sessions to that Virtual HUB, they are
automatically redirected to the VPN Server running that Virtual HUB
instance such that regardless of how many VPN Servers there are, VPN
sessions connected to the same Virtual HUB are always connected to the
same VPN Server. When no one is connected to a dynamic Virtual HUB, its
instance automatically stops running and releases the CPU and memory
reserved for it.
This system makes it possible to create a large-scale Virtual HUB
hosting service capable of hosting a large number of Virtual HUBs.
Please refer to 「10.9 Setting Up a Large Scale Virtual HUB Hosting Service」 for specific configurations.
3.9.9 Connecting to Arbitrary Servers in Static Virtual HUBs
As mentioned above, a VPN connection to a static mode Virtual HUB is
automatically load shared, so it is not possible to know which VPN
Server the connection is to until it has been established.
Virtual HUB Administrators may, however, need to connect to the
static Virtual HUB instance of an arbitrary VPN Server in a cluster for
administration purposes. In this event, when creating the connection
settings in the VPN Client or the like, designate the address of the VPN
Server and the name of the Virtual HUB to which direct connection is
sought instead of designating the cluster controller as the connection
destination VPN Server. In addition, designate the password required to
connect as an Administrator user (see 「3.4.14 Administrator Connection」 for details). This
exception makes it possible for a VPN connection to be made directly to
the desired VPN Server's static Virtual HUB without going via the
cluster controller router.
3.9.10 Collectively Administering the Entire Cluster
Collectively Administering the Entire Cluster
Once the cluster is created, the entire VPN Server Administrator and
Virtual HUB Administrators need only make an administration connection
to the controller to be able to collectively administer the status and
VPN sessions of all of the Virtual HUBs operating in the cluster. The
administration of the VPN Server and Virtual HUBs is carried out using
the VPN Server Manager or vpncmd utility in the same manner as when not
using the clustering function.
Simply by connecting to the cluster controller, VPN Server
Administrators can administer all of the Virtual HUBs in the cluster.
Each Virtual HUB Administrator can administer the Virtual HUB for which
they have authority.
The only situations in which it is necessary for VPN Server
Administrators to make a direct administration connection to cluster
member servers other than the cluster controller are the following.
- When disengaging a cluster member server from a cluster and
returning its operating mode to a stand alone server.
- When adding or deleting a cluster member server license.
- When confirming which Virtual HUB instances (entities) are
actually operating within the cluster member servers.
- When editing the cluster member server's [Encryption and
Communication Setting] entries, obtaining the contents of the
Configuration file or acquiring the server's status.
Virtual HUB Administrators can only perform administration
connections to the cluster controller, and not to the cluster member
servers.
Local Bridge & Virtual Layer 3 Switch Settings
Local bridge and virtual layer 3 switch settings are carried out for
each VPN Server. However, entire VPN Server Administrator authority is
required for these settings. Please refer to 「3.9.12 Functions not Available Simultaneously with Clustering」 for further
details.
Changing Virtual HUB Types
After creating the Virtual HUB, the type (dynamic Virtual HUB or
static Virtual HUB) cannot be changed. As such, be sure to select a
suitable type when creating the Virtual HUB.
3.9.11 Cluster Configuration Licenses
Clustering Product Licenses
When using the clustering function, either the VPN Server 2.0
Enterprise Edition License or the VPN Server 2.0 Carrier Edition License
is required for each PacketiX VPN computer to be run as a cluster
controller or cluster member server. Without these product licenses the
clustering function can not be enabled.
Inserting a single license key into multiple cluster controllers or
cluster member servers causes a licensing error to occur while using
clustering, so be careful not to inadvertently enter the same license
key into more than one cluster controller or cluster member server.
Administering Clustering Connection Licenses
When using the clustering function, a connection license only needs to
be registered for the cluster controller. Connection licenses do not
have to be registered for each cluster member server (doing so is
meaningless).
Therefore, when purchasing a VPN Server 2.0 Enterprise Edition
License and intending to use the clustering function, it is necessary to
purchase more than one of said license, and of these, one is designated
as the Server ID and client licenses and bridge connection licenses are
then purchased based on the number required for this Server ID. Also,
that Server ID's product license key and connection license keys are to
be registered on the VPN Server to become the cluster controller.
The number of client connection licenses required is the total number of
client connection sessions capable of connecting to the entire cluster
simultaneously. Similarly, the number of bridge connection licenses
required is the total number of bridge connection sessions capable of
connecting to the entire cluster simultaneously.
| PacketiX VPN Server 2.0's internal
SecureNAT sessions, local bridge sessions, cascade sessions and
administration sessions connected from the VPN Server Manager
and vpncmd utility to administer PacketiX VPN Server 2.0 are not
subject to the above-mentioned number of client connections or
bridge connections and do not consume a connection license. |
3.9.12 Functions not Available Simultaneously with Clustering
When the clustering function is enabled, the following functions
cannot be used at the same time.
- Cascade Connections
(it is possible to receive a
cascade connection from a separate computer)
- SecureNAT
(although PacketiX VPN Server 2.0 Carrier
Edition does allow the use of SecureNAT's Virtual DHCP Server
function)
The local bridging and virtual layer 3 switch functions can be used
normally. However, local bridging and virtual layer 3 switching of
Virtual HUB instances (entities) designated as local bridge definitions
or virtual layer 3 switch virtual interface definitions can only operate
between that VPN Server on which they actually exist. In the case of
static mode Virtual HUBs, a defined static Virtual HUB instance normally
exists, in principle, on all VPN Servers. However, in the case of
dynamic hubs, there can only be one VPN Server in the cluster on which
an instance can exist at the same time so the local bridging and virtual
layer 3 switching functions are typically not available for dynamic
Virtual HUBs.
|