State of the art analysis Sample Clauses

State of the art analysis. The optimization of decision problems using ML has been leveraged in many diverse applications using reinforcement learning (RL). Here, an agent interacts with a defined environment using the operations that are made available and learns based on the rewards and penalties taken from the environment’s output [5]. The environment is mathematically modelled using the Markov decision process (MDP), a discrete-time stochastic control process used to model the set of states, actions, rewards, and penalties. A model can further define the specific behaviour of the agent, reducing its exploration tree during training and consequently reducing its learning phase cost. This is called model-based RL and can be efficient when learning complex tasks, but it can also have limited performance in case the modelled constraints are biased and limit the solution to a local optimum. The so-called model-free RL, on the other hand, has greater costs on the training phase, due to its larger exploration space, but generally leads to a model with greater performance by computing the Xxxxxxx-optimality [6]. Today, model-free RL techniques are being used the most, being more accessible owing to greater computational resources available, coupled with state-of-the-art ML techniques used to approximate the Xxxxxxx-optimality using deep neural networks (XXXx), instead of the classic dynamic programming approach. These gained the name of deep RL (DRL) and include algorithms such as deep Q-network (DQN) [7], Dueling agents [8], asynchronous actor-critic (A3C) [9], NoisyNet [10], or combinations of them (e.g., the DeepMind’s Rainbow [11]). The application of DRL for cybersecurity is gaining widespread attention and has found early usage for MTD optimization. However, these applications are limited to specific environments such as in-vehicle networks, web applications, enterprise networks, or theoretical game-theory models [12] [13]. The MTD optimization in telecommunication networks such as 5G and beyond is still limited and to be explored.

State of the art analysis. In this section, we will illustrate a state-of-the-art analysis on software products im- plementing the technologies introduced ahead.