Publications

Journals

Deep reinforcement learning for improving competitive cycling performance

Demosthenous, G., Kyriakou, M., and Vassiliades, V. – 2022

Expert Systems with Applications, 203, 117311. https://www.sciencedirect.com/science/article/abs/pii/S095741742200673X

Developing expert systems that make use of artificial intelligence (AI) to provide predictive analytics as well as targeted recommendations for decision support has been gaining momentum in recent years. Both academia and industry are looking into creating such systems to solve real-world problems and tackle specific challenges. In our work, we investigate the potential application of different machine learning approaches to solutions around competitive cycling. Specifically, we build and evaluate prediction models that are capable of accurately predicting a cyclist’s speed and heart rate using sensory information collected during bike rides. In addition, we create a recommendation module that is able to provide real-time action suggestions to cyclists regarding their posture with the goal of improving their overall performance. We achieve this using a combination of model-based reinforcement learning (RL) and deep RL. In particular, we use model-based RL to learn a “simulator” of bike rides using the prediction models and action profiles extracted from sensors placed on the cyclists’ back. We then use deep Q-learning in the simulator to extract policies that improve a cyclist’s behavior during a bike ride. Our evaluation shows that by recommending specific actions throughout the ride, cyclists can increase their overall average speed with only a minimal impact on their heart rate. The results presented in this paper constitute clear evidence that advanced AI techniques are a prime candidate for further developing intelligent solutions in competitive cycling and other similar areas.

DNS Tunneling Detection by Cache-Property-Aware Features

Ishikura, N., Kondo, D., Vassiliades, V., Iordanov, I., and Tode, H. – 2021

IEEE Transactions on Network and Service Management, 18(2): 1203-1217. https://ieeexplore.ieee.org/abstract/document/9426926/

Many enterprises are under threat of targeted attacks aiming at data exfiltration. To launch such attacks, in recent years, attackers with their malware have exploited a covert channel that abuses the domain name system (DNS) named DNS tunneling. Although several research efforts have been made to detect DNS tunneling, the existing methods rely on features that advanced tunneling techniques can easily obfuscate by mimicking legitimate DNS clients. Such obfuscation would result in data leakage. To tackle this problem, we focused on a “trace” left by DNS tunneling that cannot be easily hidden. In the context of data exfiltration by DNS tunneling, the malware connects directly to the DNS cache server and the generated DNS tunneling queries produce cache misses with absolute certainty. In this study, we propose a DNS tunneling detection method based on the cache-property-aware features. Our experiments show that one of the proposed features can efficiently characterize the DNS tunneling traffic. Furthermore, we introduce a rule-based filter and a long short-term memory (LSTM)-based filter using this proposed feature. The rule-based filter achieves a higher rate of DNS tunneling attack detection than the LSTM one, which instead detects the attack more quickly, while both maintain a low misdetection rate.

The named data networking flow filter: Towards improved security over information leakage attacks

Kondo, D., Vassiliades, V., Silverston, T., Tode, H., and Asami, T. – 2020

Computer Networks, 173: 107187. https://www.sciencedirect.com/science/article/abs/pii/S1389128619316524

Named Data Networking (NDN) has the potential to create a more secure future Internet. It is therefore crucial to investigate its vulnerabilities in order to make it safer against information leakage attacks. In NDN, malware inside an enterprise can encode confidential information into Interest names and send it to the attacker. One of the countermeasures is to inspect a name in the Interest using a name filter and identify it as legitimate or anomalous. Although the name filter can dramatically decrease the information leakage throughput per Interest, it has a serious disadvantage: it does not consider a flow of Interests. This means that the malware can not only cause information leakage, but even improve the speed of the attack by aggressively producing massive flows of malicious Interests. This paper investigates such NDN flow attacks. Our contribution is twofold. First, we present a scheme that converts an HTTP flow into the corresponding NDN flow, as to date there is no publicly available dataset of the latter. Second, we propose an NDN flow filter based on support vector machines to classify the short-term activity of NDN consumers as legitimate or anomalous. In order to obtain legitimate and anomalous flows, we use a preprocessing anomaly detection step where we mark consumers based on their long-term activity. Our results clearly show that the flow filter improves the performance of the name filter by two orders of magnitude. Thus, we expect that our approach will drastically reduce the impact of this security attack in NDN.

A Survey on Policy Search Algorithms for Learning Robot Controllers in a Handful of Trials

Chatzilygeroudis, K., Vassiliades, V., Stulp, F., Calinon, S., and Mouret – 2019

IEEE Transactions on Robotics, 36(2): 328-347. https://ieeexplore.ieee.org/document/8944013

Most policy search (PS) algorithms require thousands of training episodes to find an effective policy, which is often infeasible with a physical robot. This survey article focuses on the extreme other end of the spectrum: how can a robot adapt with only a handful of trials (a dozen) and a few minutes? By analogy with the word “big-data,” we refer to this challenge as “micro-data reinforcement learning.” In this article, we show that a first strategy is to leverage prior knowledge on the policy structure (e.g., dynamic movement primitives), on the policy parameters (e.g., demonstrations), or on the dynamics (e.g., simulators). A second strategy is to create data-driven surrogate models of the expected reward (e.g., Bayesian optimization) or the dynamical model (e.g., model-based PS), so that the policy optimizer queries the model instead of the real system. Overall, all successful micro-data algorithms combine these two strategies by varying the kind of model and prior knowledge. The current scientific challenges essentially revolve around scaling up to complex robots, designing generic priors, and optimizing the computing time.

Conferences

Exploring Model Inversion Attacks in the Black-box Setting

Dionysiou, A., Vassiliades, V. and Athanasopoulos, E. – 2023

In Proceedings on Privacy Enhancing Technologies, 1, 190-206. https://petsymposium.org/popets/2023/popets-2023-0012.pdf

Model Inversion (MI) attacks, that aim to recover semantically meaningful reconstructions for each target class, have been extensively studied and demonstrated to be successful in the white-box setting. On the other hand, black-box MI attacks demonstrate low performance in terms of both effectiveness, ie, reconstructing samples which are identifiable as their ground-truth, and efficiency, ie, time or queries required for completing the attack process. Whether or not effective and efficient black-box MI attacks can be conducted on complex targets, such as Convolutional Neural Networks (CNNs), currently remains unclear.
In this paper, we present a feasibility study in regards to the effectiveness and efficiency of MI attacks in the black-box setting. In this context, we introduce Deep-BMI (Deep Black-box Model Inversion), a framework that supports various black-box optimizers for conducting MI attacks on deep CNNs used for image recognition. Deep-BMI’s most efficient optimizer is based on an adaptive hill climbing algorithm, whereas its most effective optimizer is based on an evolutionary algorithm capable of performing an all-class attack and returning a diversity of images in a single run. For assessing the severity of this threat, we utilize all three evaluation approaches found in the literature. In particular, we (a) conduct a user study with human participants,(b) demonstrate our actual reconstructions along with their ground-truth, and (c) use relevant quantitative metrics. Surprisingly, our results suggest that blackbox MI attacks, and for complex models, are comparable, in some cases, to those reported so far in the white-box setting.

HoneyGen: Generating Honeywords Using Representation Learning

Dionysiou, A., Vassiliades, V. and Athanasopoulos, E. – 2021

In ASIA CCS ’21: Proceedings of the 2021 ACM Asia Conference on Computer and Communications Security, 265-279. https://doi.org/10.1145/3433210.3453092

Honeywords are false passwords injected in a database for detecting password leakage. Generating honeywords is a challenging problem due to the various assumptions about the adversary’s knowledge as well as users’ password-selection behaviour. The success of a Honeywords Generation Technique (HGT) lies on the resulting honeywords; the method fails if an adversary can easily distinguish the real password. In this paper, we propose HoneyGen, a practical and highly robust HGT that produces realistic looking honeywords. We do this by leveraging representation learning techniques to learn useful and explanatory representations from a massive collection of unstructured data, i.e., each operator’s password database. We perform both a quantitative and qualitative evaluation of our framework using the state-of-the-art metrics. Our results suggest that HoneyGen generates high-quality honeywords that cause sophisticated attackers to achieve low distinguishing success rates.

Cache-Property-Aware Features for DNS Tunneling Detection

Ishikura, N., Kondo, D., Iordanov, I., Vassiliades, V. and Tode, H. – 2020

Presented at the 23rd Conference on Innovation in Clouds, Internet and Networks (ICIN 2020). https://ieeexplore.ieee.org/document/9059472

Most policy search (PS) algorithms require thousands of training episodes to find an effective policy, which is often A lot of enterprises are under threat of targeted attacks causing data exfiltration. As a means of performing the attacks, attackers and their malware have exploited DNS tunneling in recent years. Although there are many research efforts to detect DNS tunneling, the previously proposed methods rely on features that the malicious entities can easily obfuscate by mimicking legitimate ones. Therefore, this obfuscation would result in data leakage. In order to mitigate this issue, we focus on a trace of DNS tunneling, which cannot be easily hidden. In the context of DNS data exfiltration, malware connects directly to the DNS cache server, and a DNS tunneling query produces a cache miss with absolute certainty. In this work, we propose features derived from this cache property. Our extensive experiments show that one of the proposed features can clearly distinguish DNS tunneling traffic, which makes it useful to design and implement a solid DNS firewall against DNS tunneling.

Workshops

Continual Learning on the Edge with TensorFlow Lite

Demosthenous, G. and Vassiliades, V. – 2021

Findings of the CVPR 2021 Workshop on Continual Learning in Computer Vision.  https://arxiv.org/abs/2105.01946

Deploying sophisticated deep learning models on embedded devices with the purpose of solving real-world problems is a struggle using today’s technology. Privacy and data limitations, network connection issues, and the need for fast model adaptation are some of the challenges that constitute today’s approaches unfit for many applications on the edge and make real-time on-device training a necessity. Google is currently working on tackling these challenges by embedding an experimental transfer learning API to their TensorFlow Lite, machine learning library. In this paper, we show that although transfer learning is a good first step for on-device model training, it suffers from catastrophic forgetting when faced with more realistic scenarios. We present this issue by testing a simple transfer learning model on the CORe50 benchmark as well as by demonstrating its limitations directly on an Android application we developed. In addition, we expand the TensorFlow Lite library to include continual learning capabilities, by integrating a simple replay approach into the head of the current transfer learning model. We test our continual learning model on the CORe50 benchmark to show that it tackles catastrophic forgetting, and we demonstrate its ability to continually learn, even under non-ideal conditions, using the application we developed. Finally, we open-source the code of our Android application to enable developers to integrate continual learning to their own smartphone applications, as well as to facilitate further development of continual learning functionality into the TensorFlow Lite environment.

Book Chapters

Quality-Diversity Optimization: a novel branch of stochastic optimization

Chatzilygeroudis, K., Cully, A., Vassiliades, V., and Mouret, J.-B. – 2021

In Black Box Optimization, Machine Learning and No-Free Lunch Theorems, Edited by: Panos Pardalos, Michael Vrahatis, Varvara Rasskazova. 109-135. http://link.springer.com/chapter/10.1007/978-3-030-66515-9_4

DTraditional optimization algorithms search for a single global optimum that maximizes (or minimizes) the objective function. Multimodal optimization algorithms search for the highest peaks in the search space that can be more than one. Quality-Diversity algorithms are a recent addition to the evolutionary computation toolbox that do not only search for a single set of local optima, but instead try to illuminate the search space. In effect, they provide a holistic view of how high-performing solutions are distributed throughout a search space. The main differences with multimodal optimization algorithms are that (1) Quality-Diversity typically works in the behavioral space (or feature space), and not in the genotypic (or parameter) space, and (2) Quality-Diversity attempts to fill the whole behavior space, even if the niche is not a peak in the fitness landscape. In this chapter, we provide a gentle introduction to Quality-Diversity optimization, discuss the main representative algorithms, and the main current topics under consideration in the community. Throughout the chapter, we also discuss several successful applications of Quality-Diversity algorithms, including deep learning, robotics, and reinforcement learning.