This area tries to bridge the gap between human and computer vision by developing algorithms that learn from data continuously and with minimal supervision. Together with several colleagues and students, I am developing active learning, novelty detection, adaptation, and discovery algorithms in this branch of research.

2017

Large-Scale Gaussian Process Inference with Generalized Histogram Intersection Kernels for Visual Recognition Tasks.
Erik Rodner and Alexander Freytag and Paul Bodesheim and Björn Fröhlich and Joachim Denzler.
International Journal of Computer Vision (IJCV). 253-280. 2017. BibTeX pdf www

2016

Fine-tuning Deep Neural Networks in Continuous Learning Scenarios.
Christoph Käding and Erik Rodner and Alexander Freytag and Joachim Denzler.
ACCV Workshop on Interpretation and Visualization of Deep Neural Nets (ACCV-WS). 2016. BibTeX pdf www supplementary
Large-scale Active Learning with Approximated Expected Model Output Changes.
Christoph Käding and Alexander Freytag and Erik Rodner and Andrea Perino and Joachim Denzler.
German Conference on Pattern Recognition (GCPR). 179-191. 2016. BibTeX pdf www code supplementary more ...

Abstract: Incremental learning of visual concepts is one step towards reaching human capabilities beyond closed-world assumptions. Besides recent progress, it remains one of the fundamental challenges in computer vision and machine learning. Along that path, techniques are needed which allow for actively selecting informative examples from a huge pool of unlabeled images to be annotated by application experts. Whereas a manifold of active learning techniques exists, they commonly suffer from one of two drawbacks: (i) either they do not work reliably on challenging real-world data or (ii) they are kernel-based and not scalable with the magnitudes of data current vision applications need to deal with. Therefore, we present an active learning and discovery approach which can deal with huge collections of unlabeled real-world data. Our approach is based on the expected model output change principle and overcomes previous scalability issues. We present experiments on the large-scale MS-COCO dataset and on a dataset provided by biodiversity researchers. Obtained results reveal that our technique clearly improves accuracy after just a few annotations. At the same time, it outperforms previous active learning approaches in academic and real-world scenarios.
Active and Continuous Exploration with Deep Neural Networks and Expected Model Output Changes.
Christoph Käding and Erik Rodner and Alexander Freytag and Joachim Denzler.
NIPS Workshop on Continual Learning and Deep Networks (NIPS-WS). 2016. BibTeX pdf www
Watch, Ask, Learn, and Improve: A Lifelong Learning Cycle for Visual Recognition.
Christoph Käding and Erik Rodner and Alexander Freytag and Joachim Denzler.
European Symposium on Artificial Neural Networks (ESANN). 2016. BibTeX pdf code presentation

2015

Active Learning and Discovery of Object Categories in the Presence of Unnameable Instances.
Christoph Käding and Alexander Freytag and Erik Rodner and Paul Bodesheim and Joachim Denzler.
IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 4343-4352. 2015. BibTeX pdf www code presentation supplementary more ...

Abstract: Current visual recognition algorithms are "hungry" for data but massive annotation is extremely costly. Therefore, active learning algorithms are required that reduce labeling efforts to a minimum by selecting examples that are most valuable for labeling. In active learning, all categories occurring in collected data are usually assumed to be known in advance and experts should be able to label every requested instance. But do these assumptions really hold in practice? Could you name all categories in every image? Existing algorithms completely ignore the fact that there are certain examples where an oracle can not provide an answer or which even do not belong to the current problem domain. Ideally, active learning techniques should be able to discover new classes and at the same time cope with queries an expert is not able or willing to label. To meet these observations, we present a variant of the expected model output change principle for active learning and discovery in the presence of unnameable instances. Our experiments show that in these realistic scenarios, our approach substantially outperforms previous active learning methods, which are often not even able to improve with respect to the baseline of random query selection.
Local Novelty Detection in Multi-class Recognition Problems.
Paul Bodesheim and Alexander Freytag and Erik Rodner and Joachim Denzler.
IEEE Winter Conference on Applications of Computer Vision (WACV). 813-820. 2015. BibTeX pdf supplementary

2014

ARTOS -- Adaptive Real-Time Object Detection System.
Björn Barz and Erik Rodner and Joachim Denzler.
arXiv preprint arXiv:1407.2721. 2014. BibTeX pdf www code more ...

Abstract: ARTOS is all about creating, tuning, and applying object detection models with just a few clicks. In particular, ARTOS facilitates learning of models for visual object detection by eliminating the burden of having to collect and annotate a large set of positive and negative samples manually and in addition it implements a fast learning technique to reduce the time needed for the learning step. A clean and friendly GUI guides the user through the process of model creation, adaptation of learned models to different domains using in-situ images, and object detection on both offline images and images from a video stream. A library written in C++ provides the main functionality of ARTOS with a C-style procedural interface, so that it can be easily integrated with any other project.
Birds of a Feather Flock Together - Local Learning of Mid-level Representations for Fine-grained Recognition.
Alexander Freytag and Erik Rodner and Joachim Denzler.
ECCV Workshop on Parts and Attributes (ECCV-WS). 2014. BibTeX pdf www code presentation
Open-vocabulary Object Retrieval.
Sergio Guadarrama and Erik Rodner and Kate Saenko and Ning Zhang and Ryan Farrell and Jeff Donahue and Trevor Darrell.
Robotics Science and Systems (RSS). 41, ISBN 978-0-9923747-0-9. 2014. Awarded with an AAAI invited talk BibTeX pdf www more ...

Abstract: In this paper, we address the problem of retrieving objects based on open-vocabulary natural language queries: Given a phrase describing a specific object, e.g., the corn flakes box, the task is to find the best match in a set of images containing candidate objects. When naming objects, humans tend to use natural language with rich semantics, including basic-level categories, fine-grained categories, and instance-level concepts such as brand names. Existing approaches to large-scale object recognition fail in this scenario, as they expect queries that map directly to a fixed set of pre-trained visual categories, e.g. ImageNet synset tags. We address this limitation by introducing a novel object retrieval method. Given a candidate object image, we first map it to a set of words that are likely to describe it, using several learned image-to-text projections. We also propose a method for handling open-vocabularies, i.e., words not contained in the training data. We then compare the natural language query to the sets of words predicted for each candidate and select the best match. Our method can combine category- and instance-level semantics in a common representation. We present extensive experimental results on several datasets using both instance-level and category-level matching and show that our approach can accurately retrieve objects based on extremely varied open-vocabulary queries. The source code of our approach will be publicly available together with pre-trained models and could be directly used for robotics applications.
Interactive Adaptation of Real-Time Object Detectors.
Daniel Göhring and Judy Hoffman and Erik Rodner and Kate Saenko and Trevor Darrell.
International Conference on Robotics and Automation (ICRA). 1282-1289. 2014. BibTeX pdf www more ...

Abstract: In the following paper, we present a framework for quickly training 2D object detectors for robotic perception. Our method can be used by robotics practitioners to quickly (under 30 seconds per object) build a large-scale real-time perception system. In particular, we show how to create new detectors on the fly using large-scale internet image databases, thus allowing a user to choose among thousands of available categories to build a detection system suitable for the particular robotic application. Furthermore, we show how to adapt these models to the current environment with just a few in-situ images. Experiments on existing 2D benchmarks evaluate the speed, accuracy, and flexibility of our system.
Selecting Influential Examples: Active Learning with Expected Model Output Changes.
Alexander Freytag and Erik Rodner and Joachim Denzler.
European Conference on Computer Vision (ECCV). 562-577. 2014. BibTeX pdf presentation supplementary more ...

Abstract: In this paper, we introduce a new general strategy for active learning. The key idea of our approach is to measure the expected change of model outputs, a concept that generalizes previous methods based on expected model change and incorporates the underlying data distribution. For each example of an unlabeled set, the expected change of model predictions is calculated and marginalized over the unknown label. This results in a score for each unlabeled example that can be used for active learning with a broad range of models and learning algorithms. In particular, we show how to derive very efficient active learning methods for Gaussian process regression, which implement this general strategy, and link them to previous methods. We analyze our algorithms and compare them to a broad range of previous active learning strategies in experiments showing that they outperform state-of-the-art on well-established benchmark datasets in the area of visual object recognition.
Exemplar-specific Patch Features for Fine-grained Recognition.
Alexander Freytag and Erik Rodner and Trevor Darrell and Joachim Denzler.
German Conference on Pattern Recognition (GCPR). 144-156. 2014. BibTeX pdf code supplementary more ...

Abstract: In this paper, we present a new approach for fine-grained recognition or subordinate categorization, tasks where an algorithm needs to reliably differentiate between visually similar categories, e.g. different bird species. While previous approaches aim at learning a single generic representation and models with increasing complexity, we propose an orthogonal approach that learns patch representations specifically tailored to every single test exemplar. Since we query a constant number of images similar to a given test image, we obtain very compact features and avoid large-scale training with all classes and examples. Our learned mid-level features are build on shape and color detectors estimated from discovered patches reflecting small highly discriminative structures in the queried images. We evaluate our approach for fine-grained recognition on the CUB-2011 birds dataset and show that high recognition rates can be obtained by model combination.
Asymmetric and Category Invariant Feature Transformations for Domain Adaptation.
Judy Hoffman and Erik Rodner and Jeff Donahue and Brian Kulis and Kate Saenko.
International Journal of Computer Vision (IJCV). 109(1-2): 28-41. 2014. BibTeX pdf www more ...

Abstract: We address the problem of visual domain adaptation for transferring object models from one dataset or visual domain to another. We introduce a unified flexible model for both supervised and semi-supervised learning that allows us to learn transformations between domains. Additionally, we present two instantiations of the model, one for general feature adaptation/alignment, and one specifically designed for classification. First, we show how to extend metric learning methods for domain adaptation, allowing for learning metrics independent of the domain shift and the final classifier used. Furthermore, we go beyond classical metric learning by extending the method to asymmetric, category independent transformations. Our framework can adapt features even when the target domain does not have any labeled examples for some categories, and when the target and source features have different dimensions. Finally, we develop a joint learning framework for adaptive classifiers, which outperforms competing methods in terms of multi-class accuracy and scalability. We demonstrate the ability of our approach to adapt object recognition models under a variety of situations, such as differing imaging conditions, feature types, and codebooks. The experiments show its strong performance compared to previous approaches and its applicability to large-scale scenarios.

2013

Semi-Supervised Domain Adaptation with Instance Constraints.
Jeff Donahue and Judy Hoffman and Erik Rodner and Kate Saenko and Trevor Darrell.
IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 668 - 675. 2013. BibTeX pdf
Transform-based Domain Adaptation for Big Data.
Erik Rodner and Judy Hoffman and Jeff Donahue and Trevor Darrell and Kate Saenko.
NIPS Workshop on New Directions in Transfer and Multi-Task Learning (NIPS-WS). 2013. abstract version of arXiv:1308.4200 BibTeX pdf more ...

Abstract: Images seen during test time are often not from the same distribution as images used for learning. This problem, known as domain shift, occurs when training classifiers from object-centric internet image databases and trying to apply them directly to scene understanding tasks. The consequence is often severe performance degradation and is one of the major barriers for the application of classi- fiers in real-world systems. In this paper, we show how to learn transform-based domain adaptation classifiers in a scalable manner. The key idea is to exploit an implicit rank constraint, originated from a max-margin domain adaptation formulation, to make optimization tractable. Experiments show that the transformation between domains can be very efficiently learned from data and easily applied to new categories
Approximations of Gaussian Process Uncertainties for Visual Recognition Problems.
Paul Bodesheim and Alexander Freytag and Erik Rodner and Joachim Denzler.
Scandinavian Conference on Image Analysis (SCIA). 182-194. 2013. BibTeX pdf
An Efficient Approximation for Gaussian Process Regression Paul Bodesheim and Alexander Freytag and Erik Rodner and Joachim Denzler. (2013) Technical Report TR-FSU-INF-CV-2013-01 BibTeX pdf
One-class Classification with Gaussian Processes.
Michael Kemmler and Erik Rodner and Esther-Sabrina Wacker and Joachim Denzler.
Pattern Recognition. 3507-3518. 2013. BibTeX pdf
I Want To Know More: Efficient Multi-Class Incremental Learning Using Gaussian Processes.
Alexander Lütz and Erik Rodner and Joachim Denzler.
Pattern Recognition and Image Analysis. Advances in Mathematical Theory and Applications (PRIA). 23(3): 402-407. 2013. BibTeX pdf
Efficient Learning of Domain-invariant Image Representations.
Judy Hoffman and Erik Rodner and Jeff Donahue and Trevor Darrell and Kate Saenko.
International Conference on Learning Representations (ICLR). 2013. BibTeX pdf
Towards Adapting ImageNet to Reality: Scalable Domain Adaptation with Implicit Low-rank Transformations.
Erik Rodner and Judy Hoffman and Jeff Donahue and Trevor Darrell and Kate Saenko.
arXiv preprint arXiv:1308.4200. 2013. BibTeX pdf
Kernel Null Space Methods for Novelty Detection.
Paul Bodesheim and Alexander Freytag and Erik Rodner and Michael Kemmler and Joachim Denzler.
IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 3374-3381. 2013. BibTeX pdf www code presentation
Scalable Transform-based Domain Adaptation.
Erik Rodner and Judy Hoffman and Jeff Donahue and Trevor Darrell and Kate Saenko.
ICCV Workshop on Visual Domain Adaptation (ICCV-WS). 2013. BibTeX pdf
Beyond the closed-world assumption: The importance of novelty detection and open set recognition.
Joachim Denzler and Erik Rodner and Paul Bodesheim and Alexander Freytag.
GCPR Workshop on Unsolved Problems in Pattern Recognition (GCPR-WS). 2013. BibTeX pdf
Labeling examples that matter: Relevance-Based Active Learning with Gaussian Processes.
Alexander Freytag and Erik Rodner and Paul Bodesheim and Joachim Denzler.
German Conference on Pattern Recognition (GCPR). 282-291. 2013. BibTeX pdf code supplementary more ...

Abstract: Active learning is an essential tool to reduce manual annotation costs in the presence of large amounts of unsupervised data. In this paper, we introduce new active learning methods based on measuring the impact of a new example on the current model. This is done by deriving model changes of Gaussian process models in closed form. Furthermore, we study typical pitfalls in active learning and show that our methods automatically balance between the exploitation and the exploration trade-off. Experiments are performed with established benchmark datasets for visual object recognition and show that our new active learning techniques are able to outperform state-of-the-art methods.

Supplementary Material

2012

Lernen mit wenigen Beispielen für die visuelle Objekterkennung.
Erik Rodner.
Ausgezeichnete Informatikdissertationen 2011. 2012. in german BibTeX pdf www
Divergence-Based One-Class Classification Using Gaussian Processes.
Paul Bodesheim and Erik Rodner and Alexander Freytag and Joachim Denzler.
British Machine Vision Conference (BMVC). 50.1-50.11. 2012. http://dx.doi.org/10.5244/C.26.50 BibTeX pdf presentation

2011

Learning with Few Examples for Binary and Multiclass Classification Using Regularization of Randomized Trees.
Erik Rodner and Joachim Denzler.
Pattern Recognition Letters. 32(2): 244-251. 2011. BibTeX pdf
Efficient Multi-Class Incremental Learning Using Gaussian Processes.
Alexander Lütz and Erik Rodner and Joachim Denzler.
Open German-Russian Workshop on Pattern Recognition and Image Understanding (OGRW). 182-185. 2011. BibTeX pdf more ...

Abstract: One of the main assumptions in machine learning is that sufficient training data is available in advance and batch learning can be applied. However, because of the dynamics in a lot of applications, this assumption will break down in almost all cases over time. Therefore, classifiers have to be able to adapt themselves when new training data from existing or new classes becomes available, training data is changed or should be even removed. In this paper, we present a method allowing efficient incremental learning of a Gaussian process classifier. Experimental results show the benefits in terms of needed computation times compared to building the classifier from the scratch.
Learning from Few Examples for Visual Recognition Problems Erik Rodner. (2011) BibTeX pdf www

2010

One-Class Classification with Gaussian Processes.
Michael Kemmler and Erik Rodner and Joachim Denzler.
Asian Conference on Computer Vision (ACCV). 489-500. 2010. BibTeX pdf presentation
One-Shot Learning of Object Categories using Dependent Gaussian Processes.
Erik Rodner and Joachim Denzler.
Annual Symposium of the German Association for Pattern Recognition (DAGM). 232-241. 2010. BibTeX pdf

2009

Learning with Few Examples by Transferring Feature Relevance.
Erik Rodner and Joachim Denzler.
Annual Symposium of the German Association for Pattern Recognition (DAGM). 252-261. 2009. BibTeX pdf

2008

Learning with Few Examples using a Constrained Gaussian Prior on Randomized Trees.
Erik Rodner and Joachim Denzler.
Vision, Modelling, and Visualization Workshop (VMV). 159-168. 2008. BibTeX pdf