Regressors

M_KRR

template<class K = DM_Function_Base&>
class M_KRR : public M_Tadah_Base, public M_KRR_Train<DM_Function_Base&>

Kernel Ridge Regression (KRR) with Empirical Kernel Map (EKM).

This class performs Kernel Ridge Regression using an Empirical Kernel Map to efficiently handle high-dimensional data. EKM is used to map objects into a kernel feature space, where linear methods can be applied.

Empirical Kernel Map (EKM):

• Maps sample objects into finite-dimensional vectors in the kernel feature space.

• Requires a kernel and basis samples to project new samples into the space defined by these bases.

• Facilitates the kernelization of algorithms traditionally operating on vectors.

• Supports basis sample selection through methods like random sampling or finding linearly independent subsets.

Usage:

• Kernels: Use kernel functions (e.g., linear, RBF) to transform data for effective modeling in higher-dimensional spaces.

Configuration Options:

• LAMBDA: Controls regularization. Use 0 for ordinary least squares, positive values for manual setting, or -1 for automatic tuning.

• SBASIS: Defines the number of basis functions for nonlinear kernels.

Template Parameters:

K – DM_Kern_Base child, Kernel function

Kernels

Kern_Linear

class Kern_Linear : public virtual Kern_Base

Linear kernel also knows as dot product kernel

Defined for two vectors x and y:

\[ K(\mathbf{x}, \mathbf{y}) = \mathbf{x}^T \mathbf{y} = \sum_i x^{(i)} y^{(i)} \]

See also

Kern_Base BF_Linear

Subclassed by DM_Kern_Linear

Kern_LQ

class Kern_LQ : public virtual Kern_Base

Linear + Quadratic kernel

Defined for two vectors x and y:

\[ K(\mathbf{x}, \mathbf{y}) = \mathbf{x}^T \mathbf{y} + \Big(\mathbf{x}^T \mathbf{y} \Big)^2 = \sum_i x^{(i)} y^{(i)} + \Big(\sum_i x^{(i)} y^{(i)} \Big)^2 \]

See also

Kern_Base Kern_Linear Kern_Quadratic

Subclassed by DM_Kern_LQ

Kern_Polynomial

class Kern_Polynomial : public virtual Kern_Base

Polynomial kernel.

Defined for two vectors x and y:

\[ K(\mathbf{x}, \mathbf{y}) = \big( \gamma*\mathbf{x}^T \mathbf{y} + \mathrm{c} \big)^{\mathrm{d}} \]

Required Config key: MPARAMS <int> d <double> \( \gamma \) c

See also

Kern_Base

Subclassed by DM_Kern_Polynomial

Kern_Quadratic

class Kern_Quadratic : public virtual Kern_Base

Quadratic kernel - special case of 2nd order polynomial kernel

Defined for two vectors x and y:

\[ K(\mathbf{x}, \mathbf{y}) = \Big( \mathbf{x}^T \mathbf{y} \Big)^2 = \Big( \sum_i x^{(i)} y^{(i)} \Big)^2 \]

See also

Kern_Base

Subclassed by DM_Kern_Quadratic

Kern_RBF

class Kern_RBF : public virtual Kern_Base

Radial Basis Function kernel.

Defined for two vectors x and y:

\[ K(\mathbf{x}, \mathbf{y}) = \exp\Big( -\frac{||\mathbf{x}-\mathbf{y}||^2} {2\sigma^2} \Big) \]

Required Config key: MPARAMS <double> \( \sigma \)

See also

Kern_Base Kern_Linear Kern_Quadratic

Subclassed by DM_Kern_RBF

Kern_Sigmoid

class Kern_Sigmoid : public virtual Kern_Base

Sigmoid kernel.

Defined for two vectors x and y:

\[ K(\mathbf{x}, \mathbf{y}) = \tanh\big( \gamma*\mathbf{x}^T \mathbf{y} + \mathrm{c} \big) \]

Required Config key: MPARAMS <double> \( \gamma \) c

See also

Kern_Base

Subclassed by DM_Kern_Sigmoid

M_BLR

template<class BF = DM_Function_Base&>
class M_BLR : public M_Tadah_Base, public M_BLR_Train<DM_Function_Base&>

Bayesian Linear Regression (BLR).

This class implements Bayesian Linear Regression, a statistical method to make predictions using linear models with both linear and nonlinear features.

Model Supported Training Modes:

• LINEAR: Uses Ordinary Least Squares or Ridge Regression for linear relationships.

• NONLINEAR: Utilizes basis functions to handle nonlinear input spaces, transforming input descriptors into higher-dimensional feature spaces. For example, polynomial transformations.

Prediction:

• Computes predictions as a weighted sum of basis functions applied to input vectors.

Training:

• Employs regularized least squares, allowing for optional regularization through the \(\lambda\) parameter.

• Ordinary Least Squares (OLS) is a special case when \(\lambda = 0\).

Configuration Options:

• LAMBDA: Set to 0 for OLS, a positive value for specified regularization, or -1 for automatic tuning using evidence approximation.

Template Parameters:

BF – DM_BF_Base child, Basis function

Basis Functions

BF_Linear

struct BF_Linear : public virtual BF_Base

Linear basis function for modeling linear relationships.

This structure provides methods for predicting energy and forces using a linear basis function.

Subclassed by DM_BF_Linear

BF_Polynomial2

struct BF_Polynomial2 : public virtual BF_Base

Polynomial order 2 basis function with coefficient c=0.

This structure provides methods for predicting energy and forces using a second-order polynomial basis function.

Subclassed by DM_BF_Polynomial2