1. A cross-benchmark examination of feature-based algorithm selector generalization in single-objective numerical optimizationGjorgjina Cenikj, Gašper Petelin, Tome Eftimov, 2024, original scientific article Abstract: The task of selecting the best optimization algorithm for a particular problem is known as algorithm selection (AS). This involves training a model using landscape characteristics to predict algorithm performance, but a key challenge remains: making AS models generalize effectively to new, untrained benchmark suites. This study assesses AS models’ generalizability in single-objective numerical optimization across diverse benchmark suites. Using Exploratory Landscape Analysis (ELA) and transformer-based (TransOpt) features, the research investigates their individual and combined effectiveness in AS across four benchmarks: BBOB, AFFINE, RANDOM, and ZIGZAG. AS models perform differently based on benchmark suite similarities in algorithm performance distributions and single-best solvers. When suites align, these models underperform against a baseline predicting mean algorithm performance; yet, they outperform the baseline when suites differ in performance distributions and solvers. The AS models trained using the ELA landscape features are better than the models trained using the TransOpt features on the BBOB and AFFINE benchmark suites, while the opposite is true for the RANDOM benchmark suite. Ultimately, the study reveals challenges in accurately capturing algorithm performance through problem landscape features (ELA or TransOpt), impacting AS model applicability.
Keywords: algorithm selection, multi-target regression, generalization, benchmarking
Published in DiRROS: 21.05.2024; Views: 23; Downloads: 13
 Full text (2,49 MB)
This document has many files! More... |
2. NutriGreen image dataset : a collection of annotated nutrition, organic, and vegan food productsJan Drole, Igor Pravst, Tome Eftimov, Barbara Koroušić-Seljak, 2024, original scientific article Abstract: In this research, we introduce the NutriGreen dataset, which is a collection of images representing branded food products aimed for training segmentation models for detecting various labels on food packaging. Each image in the dataset comes with three distinct labels: one indicating its nutritional quality using the Nutri-Score, another denoting whether it is vegan or vegetarian origin with the V-label, and a third displaying the EU organic certification (BIO) logo.
Published in DiRROS: 23.04.2024; Views: 95; Downloads: 38
Full text (2,84 MB) |
3. |
4. PS-AAS : portfolio selection for automated algorithm selection in black-box optimizationAna Kostovska, Gjorgjina Cenikj, Diederick Vermetten, Anja Janković, Ana Nikolikj, Urban Škvorc, Peter Korošec, Carola Doerr, Tome Eftimov, 2023, published scientific conference contribution Keywords: automated algorithm selection, portfolio selection, black box optimization
Published in DiRROS: 11.12.2023; Views: 354; Downloads: 139
Full text (1,90 MB)
This document has many files! More... |
5. Sensitivity analysis of RF+clust for leave-one-problem-out performance predictionAna Nikolikj, Michal Pluhacek, Carola Doerr, Peter Korošec, Tome Eftimov, 2023, published scientific conference contribution Keywords: automated performance prediction, autoML, single-objective black-box optimization, zero-shot learning
Published in DiRROS: 13.11.2023; Views: 359; Downloads: 211
Full text (4,94 MB)
This document has many files! More... |
6. MsGEN : measuring generalization of nutrient value prediction across different recipe datasetsGordana Ispirova, Tome Eftimov, Sašo Džeroski, Barbara Koroušić-Seljak, 2023, original scientific article Abstract: In this study, we estimate the generalization of the performance of previously proposed predictive models for nutrient value prediction across different recipe datasets. For this purpose, we introduce a quantitative indicator that determines the level of generalization of using the developed predictive model for new unseen data not presented in the training process. On a predefined corpus of recipe embeddings from six publicly available recipe datasets (i.e., projecting them in the same meta-feature vector space), we train predictive models on one of the six recipe datasets and test the models on the rest of the datasets. In parallel, we define and calculate generalizability indexes which are numbers that indicate how generalizable a predictive model is i.e., how well will a predictive model learned on one dataset perform on another one not involved in the training. The evaluation results prove the validity of these indexes – their relation with the accuracy of the predictions. Further, we define three sampling techniques for selecting representative data instances that will cover all parts from the feature space uniformly (involving data from all datasets) and further will improve the generalization of a predictive model. We train predictive models with these generalized datasets and test them on instances from the six recipe datasets that are not selected and included in the generalized datasets. The results from the evaluation of these predictive models show improvement compared to the results from the predictive models trained on one recipe dataset and tested on the others separately.
Keywords: ML pipeline, predictive modeling, nutrient prediction, recipe datasets
Published in DiRROS: 25.09.2023; Views: 428; Downloads: 195
Full text (3,27 MB)
This document has many files! More... |
7. Algorithm instance footprint : separating easily solvable and challenging problem instancesAna Nikolikj, Sašo Džeroski, Mario Andrés Muñoz, Carola Doerr, Peter Korošec, Tome Eftimov, 2023, published scientific conference contribution Keywords: black-box optimization, algorithms, problem instances, machine learning
Published in DiRROS: 15.09.2023; Views: 310; Downloads: 202
Full text (2,03 MB)
This document has many files! More... |
8. Assessing the generalizability of a performance predictive modelAna Nikolikj, Gjorgjina Cenikj, Gordana Ispirova, Diederick Vermetten, Ryan Dieter Lang, Andries Petrus Engelbrecht, Carola Doerr, Peter Korošec, Tome Eftimov, 2023, published scientific conference contribution Keywords: algorithms, predictive models, machine learning
Published in DiRROS: 15.09.2023; Views: 328; Downloads: 214
Full text (935,67 KB)
This document has many files! More... |
9. |
10. DynamoRep : trajectory-based population dynamics for classification of black-box optimization problemsGjorgjina Cenikj, Gašper Petelin, Carola Doerr, Peter Korošec, Tome Eftimov, 2023, published scientific conference contribution Keywords: black-box single-objective optimization, optimization problem classification, problem representation, meta-learning
Published in DiRROS: 30.08.2023; Views: 346; Downloads: 236
Full text (650,13 KB)
This document has many files! More... |
