Wlodzislaw Duch:
A list of talks and
links to some presentations |
conferences, past and planned
Chronological list of papers
| DuchLab projects.
Google Profile |
arxiv page |
ORCID ID |
CV.
k-separability generalizes the concept of separability, making the goal of learning much easier. Networks should transfer the image of the input data into distributions that can be handled directly.
Solving highly non-separable problems is a challenge. k-separability index can characterize complexity of such problems and projection pursuit networks can find useful projections of data that have large pure clusters.
Support Feature Machine is an algorithm that searches for good features that facilitate problem solving. Combination of different types of features increases chances to find simple solutions, so the method works better that kernel SVM, and one can optimize local kernels.
Universal Learning Machine (ULM) should find the simplest data model for arbitrary data distributions, avoiding specific biases that could make them suitable for specific kind of problems.
Generation and filtering of useful random projections lead to algorithms that have better biological justification than classical MLP, are faster, easier to train and may in practice solve nonseparable problems of higher complexity than typical feedforward neural networks.
Non-Euclidean distance function replacing sigmoidal activation function offer a natural generalization of the standard MLP model, providing more flexible decision borders. An alternative way leading to similar results is based on renormalization of the input vectors using nonEuclidean norms in extended feature spaces. Both approaches influence the shapes of decision borders dramatically, allowing to reduce the complexity of MLP networks.
Surprisingly many cheap classification methods work very well on most of benchmark data. We have identified such cases and they should not be used when comparing new algorithms.
Variable Step Search Training is based on numerical techniques for optimization of neural network parameters. It is simple to implement and more efficient than backpropagation.
Error surfaces determine the efficiency of network training; they can be visualized in PCA space of the weigh sequences; surprisingly we have never seen local minima, only large plateaus.
Learning Vector Quantization methods may be improved in many ways.
Support Vector Neural Training, like SVM, starts from all data but near the end of the training use only a small subset of vectors near the decision border.
Ontogenic neural networks adjust their structure to the complexity of the data, growing, prunning and merging node functions.
Training of neural networks may be improved using global optimization methods, statistical approaches, and better initialization.
Floating Gaussian Mapping was inspired by the use of Gaussian function to calculate probability density in quantum physics; the model is equivalent to Radial Basis Function network. Early papers from the beginning of 1990. This also resulted in papers on neural versions of minimal distance methods, that was later developed into general approach to similarity-based methods.
Some complexity of neural networks may be hidden in more sophisticated transfer functions, not just sigmoids or Gaussians. There is a hierarchy of neural-like models from simple nodes implementing functions with a single parameter and fixed weighted connections, to complex agents implementing various functions and procedures and exchanging information sending messages and forming ad-hoc subnetworks. Initially this subject was ignored but now it has gained popularity.
Main papers comparing neural methods to other rule-based methods. Results have been presented in many tutorials at top conferences -- IJCNN (2001, 2000), WCCI (2002), ICONIP (2000), ICANN (2002, 2001), NNSC (2000), and in book chapters and conference papers.
MLP2LN algorithm, constrained multilayer backpropagation networks converted to a set of logical rules.
Operating in the space of all available data transformations and optimization techniques meta-learning algorithms use meta-knowledge about learning processes automatically extracted from experience of solving diverse problems, searching for most interesting compositions of transformations and optimizations, uncovering various aspects of knowledge hidden in the data. Meta-learning shifts the focus of the whole CI field from individual learning algorithms to the higher level of learning how to learn.This idea was introduced first in the framework of similarity-based methods in 2001.
The book focuses on different variants of decision tree induction but also describes the meta-learning approach in general which is applicable to other types of machine learning algorithms. The book discusses different variants of decision tree induction and represents a useful source of information to readers wishing to review some of the techniques used in decision tree learning, as well as different ensemble methods that involve decision trees. It is shown that the knowledge of different components used within decision tree learning needs to be systematized to enable the system to generate and evaluate different variants of machine learning algorithms with the aim of identifying the top-most performers or potentially the best one. A unified view of decision tree learning enables to emulate different decision tree algorithms simply by setting certain parameters. As meta-learning requires running many different processes with the aim of obtaining performance results, a detailed description of the experimental methodology and evaluation framework is provided. Meta-learning is discussed in great detail in the second half of the book. The exposition starts by presenting a comprehensive review of many meta-learning approaches explored in the past described in literature, including for instance approaches that provide a ranking of algorithms. The approach described can be related to other work that exploits planning whose aim is to construct data mining workflows. The book stimulates interchange of ideas between different, albeit related, approaches.
Computational Intelligence (CI) community has developed hundreds of algorithms for intelligent data analysis, but still many hard problems in computer vision, signal processing or text and multimedia understanding, problems that require deep learning techniques, are open.
Modern data mining packages contain numerous modules for data acquisition, pre-processing, feature selection and construction, instance selection, classification, association and approximation methods, optimization techniques, pattern discovery, clusterization, visualization and post-processing. A large data mining package allows for billions of ways in which these modules can be combined. No human expert can claim to explore and understand all possibilities in the knowledge discovery process.
This is where algorithms that learn how to learn come to rescue.
Operating in the space of all available data transformations and optimization techniques these algorithms use meta-knowledge about learning processes automatically extracted from experience of solving diverse problems. Inferences about transformations useful in different contexts help to construct learning algorithms that can uncover various aspects of knowledge hidden in the data. Meta-learning shifts the focus of the whole CI field from individual learning algorithms to the higher level of learning how to learn.
This book defines and reveals new theoretical and practical trends in meta-learning, inspiring the readers to further research in this exciting field.
The book focuses on different variants of decision tree induction but also describes the meta-learning approach in general which is applicable to other types of machine learning algorithms. The book discusses different variants of decision tree induction and represents a useful source of information to readers wishing to review some of the techniques used in decision tree learning, as well as different ensemble methods that involve decision trees. It is shown that the knowledge of different components used within decision tree learning needs to be systematized to enable the system to generate and evaluate different variants of machine learning algorithms with the aim of identifying the top-most performers or potentially the best one. A unified view of decision tree learning enables to emulate different decision tree algorithms simply by setting certain parameters. As meta-learning requires running many different processes with the aim of obtaining performance results, a detailed description of the experimental methodology and evaluation framework is provided. Meta-learning is discussed in great detail in the second half of the book. The exposition starts by presenting a comprehensive review of many meta-learning approaches explored in the past described in literature, including for instance approaches that provide a ranking of algorithms. The approach described can be related to other work that exploits planning whose aim is to construct data mining workflows. The book stimulates interchange of ideas between different, albeit related, approaches.
Fuzzy version of symbolic dynamics helps to understanding dynamical systems (in particular neurodynamics) by showing trajectories. |
Development of MDS, SOM, triangular visualization and other methods.
Searching for models that serve as bridges between mind and brain. In the "mind space" model mental events are represented using neurofuzzy approach for probability density modelling. Activation of sequences of areas in the mind space represented object recognition, action and mental trajectory, providing links between neurodynamics and visualization of mental events. |
In 1992 first preprints on noisy data trees have been published by W. Duch, and a year later two important ideas were presented in preprints. First, "Syntactic and semantic information in finite systems", containing the idea of pragmatic information measure that can be used to evaluate the value of new data for cognitive systems, including knowledge based systems.
Applications of computational quantum mechanics to calculation of complex molecular properties.
First calculations of the absorption and magnetic circular dichroism spectrum of a Jahn-Teller distorted excited state of cyclopropane (Duch, Segal, 1983) and other applications.
Calculation of complex properies
Część I. Sieci neuronowe jako narzędzia przetwarzania sygnałów biomedycznych
2. Przygotowanie danych i planowanie eksperymentu
3. Wykorzystanie sieci neuronowych do przetwarzania sygnałów bioelektrycznych na przykładzie EKG
4. Przetwarzanie sygnałów ABR przy użyciu sieci neuronowych w zadaniach diagnostyki słuchu
5. Sieci neuronowe w przetwarzaniu obrazów medycznych
6. Wykorzystanie sieci neuronowych w tomografii komputerowej
7. Neuronowo-rozmyte przetwarzanie obrazów cytologicznych w diagnostyce nowotworu piersi
8. Ocena jakości ziaren i nasion przy pomocy sieci neuronowych
9. Wykorzystanie wiedzy eksperckiej w sieci RBF na przykładzie zadania segmentacji obrazów medycznych
Część II. Zastosowanie sieci neuronowych do analizy danych medycznych oraz do wspomagania procesu diagnostycznego
10. Sieci neuronowe w ocenie prawdopodobieństwa istnienia raka jajnika u kobiet z guzami przydatkowymi
11. Wykorzystanie sieci neuronowych i metody wektorów nośnych SVM w procesie rozpoznawania aktywności ruchowej pacjentów dotkniętych chorobą Parkinsona
12. Zastosowanie sieci neuronowych w analizie wyników badania spirometrycznego
13. Analiza sygnałów EEG za pomocą sztucznych sieci neuronowych
14. Wykorzystanie wielowarstwowych sieci perceptronowych dla wspomagania obrazowej diagnostyki medycznej
15. Sieci neuronowe w bioinformatyce
Część III. Wspomaganie osób niepełnosprawnych z wykorzystaniem sieci neuronowych oraz ich wykorzystanie w protetyce i terapii
16. Neuronowe systemy sterowania urządzeniami wspomagającymi samoobsługę osób niepełnosprawnych wykorzystujące ocenę sygnałów biologicznych
17. Rozpoznawanie ruchów i gestów wykonywanych ustami w obrazie wizyjnym z użyciem sieci neuronowych
18. Sieci neuronowe oparte na strukturze Hopfielda w układach wspomagania osób niedowidzących
19. Perspektywy zastosowań impulsowej sieci neuronowej w systemach maszynowego sprzęgania układu nerwowego
20. Neuronowe modele przewidujące własności biomateriałów
Część IV. Sieci neuronowe zastosowane do modelowania choroby, terapii oraz do prognozowania wyników leczenia
21. Zastosowania sieci neuronowych w analizie przeżycia
22. Samooptymalizujące sieci neuronowe jako narzędzie wspomagające w długoterminowym przewidywaniu ryzyka wystąpienia ponownego zawału serca
23. Zastosowanie sieci neuronowych w analizie rokowniczej u chorych leczonych przeszczepieniem komórek krwiotwórczych
24. Klasyfikacja i analiza bólów kręgosłupa przy pomocy sztucznych sieci neuronowych
25. Ekstrakcja reguł z sieci neuronowych
26. Modelowanie pnia mózgu
27. Sieci neuronowe w modelowaniu chorób psychicznych
Dodatek - Kompedium sieci neuronowych
In print
Posters presented at conferences in 2018
Posters presented at conferences in 2017
Posters presented at conferences in 2016
Posters presented at the Neuromania conference, Toruń 28-28.05.2016
Posters presented at the Neuronus conference, Kraków 22-24.04.2016
Remaining posters 2016
Posters presented at conferences in 2015
Neuromania, Konferencja Studentów Toruńskiej Kognitywistyki, 25-26.05.2013; posters:
Awards for inventions:
My company DuchSoft has created between 1998-2004 data mining software GhostMiner distributed in the "business intelligence" category by Fujitsu (FQS Poland). This software was based on 4 PhD thesis of my students (Norbert Jankowski, Krzysztof Grąbczewski, Rafał Adamczak and Antoine Naud). Using this software our group has been ranked at 3rd position in the NIPS 2003 Feature Selection Challenge.