Machine learning techniques

2.3.2.1 Theory

Machine learning is a branch of articial intelligence, which is about the con-struction and study of systems that can learn from data Amor et al. [9]. There exist two techniques called decision tree and Bayesian network, and they will now be explained.

Decision Tree A decision tree is composed of three basic elements:

1. decision node specifying a test attribute.

2. edge or branch corresponding to the one of the possible attribute values which means one of the test attribute outcomes.

3. leaf which is also named an answer node, contains the class to which the object belongs.

In decision trees, two major phases should be ensured:

• Building the tree Based on a given training set, a decision tree is built. It consists of selecting for each decision node the appropriate test attribute and also to dene the class labeling each leaf.

• Classication In order to classify a new instance, we start by the root of the decision tree, the we test the attribute specied by this node. The result of this test allows to move down the tree branch relative to the attribute value of the given instance. This process will be repeated until a leaf is encountered. The instance is then being classied in the same class as the one characterizing the reached leaf.

Bayesian Network Bayes networks are one of the most widely used graphical models to represent and handle uncertain information. Bayes networks are specied by two components:

1. Graphical component is composed of a directed acyclic graph(DAG) where vertices represent events and edges are relations between events.

2. Numerical component consisting in a quantication of dierent links in the DAC by a conditional probability distribution of each node in the context of its parents.

Naive Bayes are very simple Bayes networks which are composed of DAGs with only one root node (called parent), representing the unobserved node, and sev-eral children, corresponding to observed nodes, with the strong assumption of independence among child nodes in the context of their parent.

2.3.2.2 Approach

Figure 2.6: Proposed work

The processes of the proposed system (gure 2.6) is briey explained in the following Natesan et al. [10]:

• Process 1: Preprocessing: For each network connection,the following three major groups of features for detecting intrusions are extracted. They are Basic features, Content features and Trac features.

• Process 2: Instance Labeling: After extracting KDDCup 99 features from each record, the instances are labeled as Normal or any one of the attack category such as Dos, Probe, R2L and U2R.

• Process 3: Selection of weak classiers: The various weak classiers used in their proposed system are Naive Bayes, Bayes Net and Decision Tree.

They have used the single weak classier along with the boosting algorithm to improve the classication accuracy.

• Process 4: Combining weak classiers: In order to improve the classica-tion accuracy further it has been proposed to combine two weak classiers along with the boosting algorithm.

• Process 5: Building of strong classier: A strong classier is constructed by combining two weak classiers and boosting algorithm. The strong classier results in higher attack detection rate than single weak classier.

Results: The overall detection rate and false alarm rate of the three single weak classiers are shown in gure 2.7. Decision tree was able to give a high detection rate in the case of DoS and Probe attacks and the Naive Bayes algorithm with Adaboost (AdaBoost is a machine learning algorithm, can be used in conjunction with many other learning algorithms to improve their performance. It calls a weak classier repeatedly in a series of rounds.) detects the R2L and U2R attacks comparatively better than other algorithms.

Figure 2.7: The attack detection rate of dierent weak classiers

Figure 2.8: The false alarm rate of dierent weak classiers

Figure 2.9: The attack detection rate of dierent combinations of weak clas-siers

The detection rate of the various attack categories by using the three dierent combinations of weak classiers with the Adaboost algorithm shown in gure 2.9. It can be seen that, the performance of NB-DT combination with the Adaboost algorithm is comparatively better than the other two combinations of weak classiers.

Figure 2.10: False alarm rate comparison

The false alarm rate of BN-NB combination of weak classier with Adaboost decreases to2.12%, but it shows an increase in the case of BN-DT and NB-DT combinations of weak classiers as shown in gure 2.10. The training time and the testing time of various combinations of weak classiers with Adaboost is shown in gure 2.10. The NB-DT combination with the Adaboost took less training time and testing time than other two combinations of weak classiers.

Summary: They have proposed an Adaboost algorithm with dierent combina-tion of weak classiers. The weak classiers such as Bayes Net, Naive Bayes and Decision tree are used in three dierent combinations such as BN-NB, BN-DT and NB-DT with Adaboost algorithm to improve the classication accuracy.

The various challenges of IDS such as attack detection rate, false alarm rate and computional time for building robust, scalable and ecient system are ad-dressed. It is important to have a low false alarm rate for an IDS with higher detection rate. The experiment result shows that the NB-DT combination with Adaboost algorithm has a very low false-alarm rate with a high detection rate.

They have focused mainly to obtain better classication though the time and computational complexities are theoretically high. But practically the time and computational complexities are reduced by processing speed of the computing device.

Figure 2.11: Comparison with other algorithms