Training of ATS-SVMs for multi-category classification. More...
#include <shark/Algorithms/Trainers/McSvmATSTrainer.h>
Inheritance diagram for shark::McSvmATSTrainer< InputType, CacheType >:
Public Types | |
| typedef CacheType | QpFloatType |
| typedef AbstractModel< InputType, RealVector > | ModelType |
| typedef AbstractKernelFunction< InputType > | KernelType |
| Public Types inherited from shark::AbstractSvmTrainer< InputType, unsigned int > | |
| typedef AbstractKernelFunction< InputType > | KernelType |
| Public Types inherited from shark::AbstractTrainer< Model, LabelTypeT > | |
| typedef Model | ModelType |
| typedef ModelType::InputType | InputType |
| typedef LabelTypeT | LabelType |
| typedef LabeledData< InputType, LabelType > | DatasetType |
Public Member Functions | |
| McSvmATSTrainer (KernelType *kernel, double C, bool unconstrained=false) | |
| std::string | name () const |
| From INameable: return the class name. More... | |
| void | train (KernelClassifier< InputType > &svm, const LabeledData< InputType, unsigned int > &dataset) |
| Public Member Functions inherited from shark::AbstractSvmTrainer< InputType, unsigned int > | |
| AbstractSvmTrainer (KernelType *kernel, double C, bool offset, bool unconstrained=false) | |
| AbstractSvmTrainer (KernelType *kernel, double negativeC, double positiveC, bool offset, bool unconstrained=false) | |
| double | C () const |
| Return the value of the regularization parameter C. More... | |
| RealVector const & | regularizationParameters () const |
| RealVector & | regularizationParameters () |
| KernelType * | kernel () |
| const KernelType * | kernel () const |
| void | setKernel (KernelType *kernel) |
| bool | isUnconstrained () const |
| bool | trainOffset () const |
| double | CacheSize () const |
| void | setCacheSize (std::size_t size) |
| RealVector | parameterVector () const |
| get the hyper-parameter vector More... | |
| void | setParameterVector (RealVector const &newParameters) |
| set the vector of hyper-parameters More... | |
| size_t | numberOfParameters () const |
| return the number of hyper-parameters More... | |
| Public Member Functions inherited from shark::AbstractTrainer< Model, LabelTypeT > | |
| virtual void | train (ModelType &model, DatasetType const &dataset)=0 |
| Core of the Trainer interface. More... | |
| Public Member Functions inherited from shark::INameable | |
| virtual | ~INameable () |
| Public Member Functions inherited from shark::ISerializable | |
| virtual | ~ISerializable () |
| Virtual d'tor. More... | |
| virtual void | read (InArchive &archive) |
| Read the component from the supplied archive. More... | |
| virtual void | write (OutArchive &archive) const |
| Write the component to the supplied archive. More... | |
| void | load (InArchive &archive, unsigned int version) |
| Versioned loading of components, calls read(...). More... | |
| void | save (OutArchive &archive, unsigned int version) const |
| Versioned storing of components, calls write(...). More... | |
| BOOST_SERIALIZATION_SPLIT_MEMBER () | |
| Public Member Functions inherited from shark::QpConfig | |
| QpConfig (bool precomputedFlag=false, bool sparsifyFlag=true) | |
| Constructor. More... | |
| QpStoppingCondition & | stoppingCondition () |
| Read/write access to the stopping condition. More... | |
| QpStoppingCondition const & | stoppingCondition () const |
| Read access to the stopping condition. More... | |
| QpSolutionProperties & | solutionProperties () |
| Access to the solution properties. More... | |
| bool & | precomputeKernel () |
| Flag for using a precomputed kernel matrix. More... | |
| bool const & | precomputeKernel () const |
| Flag for using a precomputed kernel matrix. More... | |
| bool & | sparsify () |
| Flag for sparsifying the model after training. More... | |
| bool const & | sparsify () const |
| Flag for sparsifying the model after training. More... | |
| bool & | shrinking () |
| Flag for shrinking in the decomposition solver. More... | |
| bool const & | shrinking () const |
| Flag for shrinking in the decomposition solver. More... | |
| bool & | s2do () |
| Flag for S2DO (instead of SMO) More... | |
| bool const & | s2do () const |
| Flag for S2DO (instead of SMO) More... | |
| unsigned int & | verbosity () |
| Verbosity level of the solver. More... | |
| unsigned int const & | verbosity () const |
| Verbosity level of the solver. More... | |
| unsigned long long const & | accessCount () const |
| Number of kernel accesses. More... | |
| void | setMinAccuracy (double a) |
| void | setMaxIterations (unsigned long long i) |
| void | setTargetValue (double v) |
| void | setMaxSeconds (double s) |
| Public Member Functions inherited from shark::IParameterizable | |
| virtual | ~IParameterizable () |
Additional Inherited Members | |
| Protected Attributes inherited from shark::AbstractSvmTrainer< InputType, unsigned int > | |
| KernelType * | m_kernel |
| RealVector | m_regularizers |
| Vector of regularization parameters. More... | |
| bool | m_trainOffset |
| bool | m_unconstrained |
| Is log(C) stored internally as a parameter instead of C? If yes, then we get rid of the constraint C > 0 on the level of the parameter interface. More... | |
| std::size_t | m_cacheSize |
| Number of values in the kernel cache. The size of the cache in bytes is the size of one entry (4 for float, 8 for double) times this number. More... | |
| Protected Attributes inherited from shark::QpConfig | |
| QpStoppingCondition | m_stoppingcondition |
| conditions for when to stop the QP solver More... | |
| QpSolutionProperties | m_solutionproperties |
| properties of the approximate solution found by the solver More... | |
| bool | m_precomputedKernelMatrix |
| should the solver use a precomputed kernel matrix? More... | |
| bool | m_sparsify |
| should the trainer sparsify the model after training? More... | |
| bool | m_shrinking |
| should shrinking be used? More... | |
| bool | m_s2do |
| should S2DO be used instead of SMO? More... | |
| unsigned int | m_verbosity |
| verbosity level (currently unused) More... | |
| unsigned long long | m_accessCount |
| kernel access count More... | |
Detailed Description
template<class InputType, class CacheType = float>
class shark::McSvmATSTrainer< InputType, CacheType >
Training of ATS-SVMs for multi-category classification.
The ATS-SVM is a special support vector machine variant for classification of more than two classes. Given are data tuples \( (x_i, y_i) \) with x-component denoting input and y-component denoting the label 1, ..., d (see the tutorial on label conventions; the implementation uses values 0 to d-1), a kernel function k(x, x') and a regularization constant C > 0. Let H denote the kernel induced reproducing kernel Hilbert space of k, and let \( \phi \) denote the corresponding feature map. Then the SVM classifier is the function
\[ h(x) = \arg \max (f_c(x)) \]
\[ f_c(x) = \langle w_c, \phi(x) \rangle + b_c \]
\[ f = (f_1, \dots, f_d) \]
with class-wise coefficients w_c and b_c given by the (primal) optimization problem
\[ \min \frac{1}{2} \sum_c \|w_c\|^2 + C \sum_i L(y_i, f(x_i)) \]
\[ \text{s.t. } \sum_c f_c = 0 \]
The special property of the so-called ATS machine is its loss function, which arises from the application of the total sum operator to absolute margin violations. Let \( h(m) = \max\{0, 1-m\} \) denote the hinge loss as a function of the margin m, then the ATS loss is given by
\[ L(y, f(x)) = \sum_c h((2 \cdot \delta_{c,y} - 1) \cdot f_c(x)) \]
where the Kronecker delta is one if its arguments agree and zero otherwise.
For more details refer to the technical report:
Fast Training of Multi-Class Support Vector Machines. Ü Dogan, T. Glasmachers, and C. Igel, Technical Report 2011/3, Department of Computer Science, University of Copenhagen, 2011.
Definition at line 99 of file McSvmATSTrainer.h.
Member Typedef Documentation
§ KernelType
| typedef AbstractKernelFunction<InputType> shark::McSvmATSTrainer< InputType, CacheType >::KernelType |
Definition at line 105 of file McSvmATSTrainer.h.
§ ModelType
| typedef AbstractModel<InputType, RealVector> shark::McSvmATSTrainer< InputType, CacheType >::ModelType |
Definition at line 104 of file McSvmATSTrainer.h.
§ QpFloatType
| typedef CacheType shark::McSvmATSTrainer< InputType, CacheType >::QpFloatType |
Definition at line 103 of file McSvmATSTrainer.h.
Constructor & Destructor Documentation
§ McSvmATSTrainer()
|
inline |
Constructor
- Parameters
-
kernel kernel function to use for training and prediction C regularization parameter - always the 'true' value of C, even when unconstrained is set unconstrained when a C-value is given via setParameter, should it be piped through the exp-function before using it in the solver?
Definition at line 112 of file McSvmATSTrainer.h.
Member Function Documentation
§ name()
|
inlinevirtual |
From INameable: return the class name.
Reimplemented from shark::INameable.
Definition at line 117 of file McSvmATSTrainer.h.
§ train()
|
inline |
Definition at line 120 of file McSvmATSTrainer.h.
References shark::QpSparseArray< QpFloatType >::add(), shark::AbstractSvmTrainer< InputType, unsigned int >::C(), shark::ArgMaxConverter< KernelExpansion< InputType > >::decisionFunction(), shark::LabeledData< InputT, LabelT >::element(), shark::LabeledData< InputT, LabelT >::inputs(), shark::LabeledData< InputT, LabelT >::labels(), shark::QpConfig::m_accessCount, shark::AbstractSvmTrainer< InputType, unsigned int >::m_cacheSize, shark::AbstractSvmTrainer< InputType, unsigned int >::m_kernel, shark::QpConfig::m_shrinking, shark::QpConfig::m_solutionproperties, shark::QpConfig::m_stoppingcondition, shark::AbstractSvmTrainer< InputType, unsigned int >::m_trainOffset, shark::numberOfClasses(), shark::LabeledData< InputT, LabelT >::numberOfElements(), shark::QpConfig::precomputeKernel(), shark::QpSparseArray< QpFloatType >::setDefaultValue(), shark::QpSolver< Problem, SelectionStrategy >::solve(), shark::BiasSolver< Matrix >::solve(), shark::QpConfig::sparsify(), and shark::blas::sum().
The documentation for this class was generated from the following file:
- include/shark/Algorithms/Trainers/McSvmATSTrainer.h