API
ChemometricsTools.BlandAltman
— Method.BlandAltman(Y1, Y2; Confidence = 1.96)
Returns a Plot object of a Bland-Altman plot between vectors Y1
and Y2
with a confidence limit of Confidence
.
ChemometricsTools.Bounds
— Method.Bounds(dims)
Constructor for a Bounds object. Returns a bounds object with a lower bound of [lower...] and upper bound[upper...] with length of dims
.
ChemometricsTools.Bounds
— Method.Bounds(dims)
Default constructor for a Bounds object. Returns a bounds object with a lower bound of [0...] and upper bound[1...] with length of dims
.
ChemometricsTools.CORAL
— Method.CORAL(X1, X2; lambda = 1.0)
Performs CORAL to facilitate covariance based transfer from X1
to X2
with regularization parameter lambda
. Returns a CORAL object.
Correlation Alignment for Unsupervised Domain Adaptation. Baochen Sun, Jiashi Feng, Kate Saenko. https://arxiv.org/abs/1612.01939
ChemometricsTools.CORAL
— Method.(C::CORAL)(Z)
Applies a the transform from a learned CORAL
object to new data Z
.
CanonicalCorrelationAnalysis(A, B)
Returns a CanonicalCorrelationAnalysis object which contains (U, V, r) from Arrays A and B. Currently Untested for correctness but should compute....
ChemometricsTools.Center
— Method.(T::Center)(Z; inverse = false)
Centers data in array Z
column-wise according to learned mean centers in Center object T
.
ChemometricsTools.CenterScale
— Method.(T::CenterScale)(Z; inverse = false)
Centers and Scales data in array Z
column-wise according to learned measures of central tendancy in Scale object T
.
ChemometricsTools.ClassicLeastSquares
— Method.ClassicLeastSquares( X, Y; Bias = false )
Makes a ClassicLeastSquares regression model of the form Y
= AX
with or without a Bias
term. Returns a CLS object.
ChemometricsTools.ClassicLeastSquares
— Method.(M::ClassicLeastSquares)(X)
Makes an inference from X
using a ClassicLeastSquares object.
ChemometricsTools.GaussianBand
— Type.GaussianBand(sigma,amplitude,center)
Constructs a Gaussian kernel generator.
ChemometricsTools.GaussianBand
— Method.(B::GaussianBand)(X::Float64)
Returns the scalar probability associated with a GaussianBand object (kernel) at a location in space(X
).
ChemometricsTools.KFoldsValidation
— Method.KFoldsValidation(K::Int, x, y)
Returns a KFoldsValidation iterator with K
folds. Because it's an iterator it can be used in for loops, see the tutorials for pragmatic examples. The iterator returns a 2-Tuple of 2-Tuples which have the following form: ((TrainX,TrainY),(ValidateX,ValidateY)
.
ChemometricsTools.LDA
— Method.LDA(X, Y; Factors = 1)
Compute's a LinearDiscriminantAnalysis transform from x
with a user specified number of latent variables(Factors
). Returns an LDA object.
ChemometricsTools.LDA
— Method.( model::LDA )( Z; Factors = length(model.Values) )
Calling a LDA object on new data brings the new data Z
into the LDA basis.
ChemometricsTools.LSSVM
— Method.LSSVM( X, Y, Penalty; KernelParameter = 0.0, KernelType = "linear" )
Makes a LSSVM model of the form Y
= AK
with a bias term using a user specified Kernel("linear", or "gaussian") and has an L2 Penalty
. Returns a LSSVM Wrapper for a CLS object.
ChemometricsTools.LSSVM
— Method.(M::LSSVM)(X)
Makes an inference from X
using a LSSVM object.
ChemometricsTools.LorentzianBand
— Type.LorentzianBand(gamma,amplitude,center)
Constructs a Lorentzian kernel generator.
ChemometricsTools.LorentzianBand
— Method.(B::LorentzianBand)(X::Float64)
Returns the probability associated with a LorentzianBand object (kernel) at a location in space(X
).
ChemometricsTools.MultiCenter
— Type.MultiCenter(Z, mode = 1)
Acquires the mean of the specified mode in Z
and returns a transform that will remove those means from any future data.
ChemometricsTools.MultiCenter
— Method.(T::MultiCenter)(Z; inverse = false)
Centers data in Tensor Z
mode-wise according to learned centers in MultiCenter object T
.
ChemometricsTools.MultiScale
— Type.MultiScale(Z, mode = 1)
Acquires the standard deviations of the specified mode in Z
and returns a transform that will scale by those standard deviations from any future data.
ChemometricsTools.MultiScale
— Method.(T::MultiScale)(Z; inverse = false)
Scales data in Tensor Z
mode-wise according to learned standard deviations in MultiScale object T
.
(T::MultiplicativeScatterCorrection)(Z)
Applies MultiplicativeScatterCorrection from a stored object T
to Array Z
.
OrthogonalSignalCorrection(X, Y; Factors = 1)
Performs Thomas Fearn's Orthogonal Signal Correction to an endogenous X
and exogenous Y
. The number of Factors
are the number of orthogonal components to be removed from X
. This function returns an OSC object.
Tom Fearn. On orthogonal signal correction. Chemometrics and Intelligent Laboratory Systems. Volume 50, Issue 1, 2000, Pages 47-52.
(OSC::OrthogonalSignalCorrection)(Z; Factors = 2)
Applies a the transform from a learned orthogonal signal correction object OSC
to new data Z
.
ChemometricsTools.PCA
— Method.PCA(X; Factors = minimum(size(X)) - 1)
Compute's a PCA from x
using LinearAlgebra's SVD algorithm with a user specified number of latent variables(Factors
). Returns a PCA object.
ChemometricsTools.PCA
— Method.(T::PCA)(Z::Array; Factors = length(T.Values), inverse = false)
Calling a PCA object on new data brings the new data Z
into or out of (inverse
= true) the PCA basis.
ChemometricsTools.PartialLeastSquares
— Method.PartialLeastSquares( X, Y; Factors = minimum(size(X)) - 2, tolerance = 1e-8, maxiters = 200 )
Returns a PartialLeastSquares regression model object from arrays X
and Y
.
- PARTIAL LEAST-SQUARES REGRESSION: A TUTORIAL PAUL GELADI and BRUCE R.KOWALSKI. Analytica Chimica Acta, 186, (1986) PARTIAL LEAST-SQUARES REGRESSION:
- Martens H., NÊs T. Multivariate Calibration. Wiley: New York, 1989.
- Re-interpretation of NIPALS results solves PLSR inconsistency problem. Rolf Ergon. Published in Journal of Chemometrics 2009; Vol. 23/1: 72-75
ChemometricsTools.PartialLeastSquares
— Method.(M::PartialLeastSquares)
Makes an inference from X
using a PartialLeastSquares object.
ChemometricsTools.Particle
— Method.Particle(ProblemBounds, VelocityBounds)
Default constructor for a Particle object. It creates a random unformly distributed particle within the specified ProblemBounds
, and limits it's velocity to the specified VelocityBounds
.
(M::PrincipalComponentRegression)( X )
Makes an inference from X
using a PrincipalComponentRegression object.
PrincipalComponentRegression(PCAObject, Y )
Makes a PrincipalComponentRegression model object from a PCA Object and property value Y
.
ChemometricsTools.QQ
— Method.QQ( Y1, Y2; Quantiles = collect( 1 : 99 ) ./ 100 )
Returns a Plot object of a Quantile-Quantile plot between vectors Y1
and Y2
at the desired Quantiles
.
ChemometricsTools.QuantileTrim
— Type.(T::QuantileTrim)(X, inverse = false)
Trims data in array X
columns wise according to learned quantiles in QuantileTrim object T
This function does NOT have an inverse.
ChemometricsTools.QuantileTrim
— Method.QuantileTrim(Z; quantiles::Tuple{Float64,Float64} = (0.05, 0.95) )
Trims values above or below the specified columnwise quantiles to the quantile values themselves.
ChemometricsTools.RangeNorm
— Method.(T::RangeNorm)(Z; inverse = false)
Scales and shifts data in array Z
column-wise according to learned min-maxes in RangeNorm object T
.
ChemometricsTools.RidgeRegression
— Method.RidgeRegression( X, Y, Penalty; Bias = false )
Makes a RidgeRegression model of the form Y
= AX
with or without a Bias
term and has an L2 Penalty
. Returns a CLS object.
ChemometricsTools.RidgeRegression
— Method.(M::RidgeRegression)(X)
Makes an inference from X
using a RidgeRegression object which wraps a ClassicLeastSquares object.
ChemometricsTools.RollingWindow
— Method.RollingWindow(samples::Int,windowsize::Int,skip::Int)
Creates a RollingWindow iterator from a number of samples
and a static windowsize
where every iteration skip
steps are skipped. The iterator can be used in for loops to iteratively return indices of a dynamic rolling window.
ChemometricsTools.RollingWindow
— Method.RollingWindow(samples::Int,windowsize::Int)
Creates a RollingWindow iterator from a number of samples
and a static windowsize
. The iterator can be used in for loops to iteratively return indices of a dynamic rolling window.
ChemometricsTools.RunningMean
— Method.RunningMean(x)
Constructs a running mean object with an initial scalar value of x
.
ChemometricsTools.RunningVar
— Method.RunningVar(x)
Constructs a RunningVar object with an initial scalar value of x
. Note: RunningVar objects implicitly calculate the running mean.
ChemometricsTools.Scale
— Method.(T::Scale)(Z; inverse = false)
Scales data in array Z
column-wise according to learned standard deviations in Scale object T
.
TransferByOrthogonalProjection(X1, X2; Factors = 1)
Performs Thomas Fearns Transfer By Orthogonal Projection to facilitate transfer from X1
to X2
. Returns a TransferByOrthogonalProjection object.
Anne Andrew, Tom Fearn. Transfer by orthogonal projection: making near-infrared calibrations robust to between-instrument variation. Chemometrics and Intelligent Laboratory Systems. Volume 72, Issue 1, 2004, Pages 51-56,
(TbOP::TransferByOrthogonalProjection)(X1; Factors = TbOP.Factors)
Applies a the transform from a learned transfer by orthogonal projection object TbOP
to new data X1
.
ChemometricsTools.Universe
— Method.(U::Universe)(Band...)
A Universe objects internal "spectra" can be updated to include the additive contribution of many Band-like objects.
ChemometricsTools.Universe
— Method.Universe(mini, maxi; width = nothing, bins = nothing)
Creates a 1-D discretized segment that starts at mini and ends at maxi. The width of the bins for the discretization can either be provided or inferred from the number of bins. Returns a Universe object.
ChemometricsTools.Universe
— Method.(U::Universe)(Band::Union{ GaussianBand, LorentzianBand})
A Universe objects internal "spectra" can be updated to include the additive contribution of any Band-like object.
ChemometricsTools.ALSSmoother
— Method.ALSSmoother(X; lambda = 100, p = 0.001, maxiters = 10)
Applies an assymetric least squares smoothing function to a 2-Array X
. The lambda
, p
, and maxiters
parameters control the smoothness. See the reference below for more information.
Paul H. C. Eilers, Hans F.M. Boelens. Baseline Correction with Asymmetric Least Squares Smoothing. 2005
ChemometricsTools.AssessHealth
— Method.AssessHealth( X )
Returns a somewhat detailed Dict containing information about the 'health' of a dataset. What is included is the following: - PercentMissing: percent of missing entries (includes nothing, inf / nan) in the dataset - EmptyColumns: the columns which have only 1 value - RankEstimate: An estimate of the rank of X - (optional)Duplicates: returns the rows of duplicate observations
ChemometricsTools.ClassificationTree
— Method.ClassificationTree(x, y; gainfn = entropy, maxdepth = 4, minbranchsize = 3)
Builds a CART object using either gini or entropy as a partioning method. Y must be a one hot encoded 2-Array. Predictions can be formed by calling the following function from the CART object: (M::CART)(x).
*Note: this is a purely nonrecursive decision tree. The julia compiler doesn't like storing structs of nested things. I wrote it the recursive way in the past and it was quite slow, I think this is true also of interpretted languages like R/Python...So here it is, nonrecursive tree's!
ChemometricsTools.ColdToHot
— Method.ColdToHot(Y, Schema::ClassificationLabel)
Turns a cold encoded Y
vector into a one hot encoded array.
ChemometricsTools.DirectStandardization
— Method.DirectStandardization(InstrumentX1, InstrumentX2; Factors = minimum(collect(size(InstrumentX1))) - 1)
Makes a DirectStandardization object to facilitate the transfer from Instrument #2 to Instrument #1 . The returned object can be used to transfer unseen data to the approximated space of instrument 1. The number of Factors
used are those from the internal orthogonal basis.
Yongdong Wang and Bruce R. Kowalski, "Calibration Transfer and Measurement Stability of Near-Infrared Spectrometers," Appl. Spectrosc. 46, 764-771 (1992)
ChemometricsTools.EWMA
— Method.EWMA(Initial::Float64, Lambda::Float64) = ewma(Lambda, Initial, Initial, RunningVar(Initial))
Constructs an exponentially weighted moving average object from an vector of scalar property values Initial
and the decay parameter Lambda
. This computes the running statistcs neccesary for creating the EWMA model using the interval provided and updates the center value to the mean of the provided values.
ChemometricsTools.EWMA
— Method.EWMA(Initial::Float64, Lambda::Float64) = ewma(Lambda, Initial, Initial, RunningVar(Initial))
Constructs an exponentially weighted moving average object from an initial scalar property value Initial
and the decay parameter Lambda
. This defaults the center value to be the initial value.
ChemometricsTools.EmpiricalQuantiles
— Method.EmpiricalQuantiles(X, quantiles)
Finds the column-wise quantiles
of 2-Array X
and returns them in a 2-Array of size quantiles
by variables
. *Note: This copies the array... Use a subset if memory is the concern. *
ChemometricsTools.ExplainedVariance
— Method.ExplainedVariance(lda::LDA)
Calculates the explained variance of each singular value in an LDA object.
ChemometricsTools.ExplainedVariance
— Method.ExplainedVariance(PCA::PCA)
Calculates the explained variance of each singular value in a pca object.
ChemometricsTools.ExtremeLearningMachine
— Function.ExtremeLearningMachine(X, Y, ReservoirSize = 10; ActivationFn = sigmoid)
Returns a ELM regression model object from arrays X
and Y
, with a user specified ReservoirSize
and ActivationFn
.
Extreme learning machine: a new learning scheme of feedforward neural networks. Guang-Bin Huang ; Qin-Yu Zhu ; Chee-Kheong Siew. 2004 IEEE International Joint...
ChemometricsTools.FirstDerivative
— Method.FirstDerivative(X)
Uses the finite difference method to compute the first derivative for every row in X
. Note: This operation results in the loss of a column dimension.
ChemometricsTools.FractionalDerivative
— Function.FractionalDerivative(Y, X = 1 : length(Y); Order = 0.5)
Calculates the Grunwald-Leitnikov fractional order derivative on every row of Array Y. Array X
is a vector that has the spacing between column-wise entries in Y
. X
can be a scalar if that is constant (common in spectroscopy). Order
is the fractional order of the derivative. Note: This operation results in the loss of a column dimension.
The Fractional Calculus, by Oldham, K.; and Spanier, J. Hardcover: 234 pages. Publisher: Academic Press, 1974. ISBN 0-12-525550-0
ChemometricsTools.HLDA
— Method.HLDA(X, YHOT; K = 1, Factors = 1)
Compute's a Hierarchical LinearDiscriminantAnalysis transform from x
with a user specified number of latent variables(Factors
). The adjacency matrices are created from K
nearest neighbors.
Returns an LDA object. Note: this can be used with any other LDA functions such as Gaussian discriminants or explained variance.
Lu D, Ding C, Xu J, Wang S. Hierarchical Discriminant Analysis. Sensors (Basel). 2018 Jan 18;18(1). pii: E279. doi: 10.3390/s18010279.
ChemometricsTools.HighestVote
— Method.HighestVote(yhat)
Returns the column index for each row that has the highest value in one hot encoded yhat
. Returns a one cold encoded vector.
ChemometricsTools.HighestVoteOneHot
— Method.HighestVoteOneHot(yhat)
Turns the highest column-wise value to a 1 and the others to zeros per row in a one hot encoded yhat
. Returns a one cold encoded vector.
ChemometricsTools.HotToCold
— Method.HotToCold(Y, Schema::ClassificationLabel)
Turns a one hot encoded Y
array into a cold encoded vector.
ChemometricsTools.IntervalOverlay
— Method.IntervalOverlay(Spectra, Intervals, Err)
Displays the relative error(Err
) of each interval
ontop of a Spectra
.
ChemometricsTools.IsColdEncoded
— Method.IsColdEncoded(Y)
Returns a boolean true if the array Y is cold encoded, and false if not.
ChemometricsTools.KennardStone
— Method.KennardStone(X, TrainSamples; distance = "euclidean")
Returns the indices of the Kennard-Stone sampled exemplars (E), and those not sampled (O) as a 2-Tuple (E, O).
R. W. Kennard & L. A. Stone (1969) Computer Aided Design of Experiments, Technometrics, 111, 137-148, DOI: 10.1080/00401706.1969.10490666
ChemometricsTools.KernelRidgeRegression
— Method.KernelRidgeRegression( X, Y, Penalty; KernelParameter = 0.0, KernelType = "linear" )
Makes a KernelRidgeRegression model of the form Y
= AK
using a user specified Kernel("Linear", or "Guassian") and has an L2 Penalty
. Returns a KRR Wrapper for a CLS object.
ChemometricsTools.LabelEncoding
— Method." LabelEncoding(HotOrCold)
Determines if an Array, Y
, is one hot encoded, or cold encoded by it's dimensions. Returns a ClassificationLabel object/schema to convert between the formats.
ChemometricsTools.LeaveOneOut
— Method.LeaveOneOut(x, y)
Returns a KFoldsValidation iterator with leave one out folds. Because it's an iterator it can be used in for loops, see the tutorials for pragmatic examples. The iterator returns a 2-Tuple of 2-Tuples which have the following form: ((TrainX,TrainY),(ValidateX,ValidateY)
.
ChemometricsTools.Lifeform
— Method.Lifeform(size, onlikelihood, initialscore)
Constructor for a BinaryLifeForm struct. Binary life forms are basically wrappers for a binary vector, which has a likelihood for being 1(onlikelihood
). Each life form also has a score
based on it's "fitness". So the GA's in this package can be used to minimize or maximize this is an open parameter, but Inf/-Inf is a good initialscore
.
ChemometricsTools.Limits
— Method.Limits(P::ewma; k = 3.0)
This function returns the upper and lower control limits with a k
span of variance for an EWMA object P
.
ChemometricsTools.Logit
— Method.Logit(Z; inverse = false)
Logit transforms (ln( X / (1 - X) ))
) every element in Z
. The inverse may also be applied. Warning: This can return Infs and NaNs if elements of Z are not suited to the transform
ChemometricsTools.MAE
— Method.MAE( y, yhat )
Calculates Mean Average Error from vectors Y
and YHat
ChemometricsTools.MAPE
— Method.MAPE( y, yhat )
Calculates Mean Average Percent Error from vectors Y
and YHat
ChemometricsTools.ME
— Method.ME( y, yhat )
Calculates Mean Error from vectors Y
and YHat
.
ChemometricsTools.MSE
— Method.MSE( y, yhat )
Calculates Mean Squared Error from vectors Y
and YHat
ChemometricsTools.Mean
— Method.Mean(rv::RunningMean)
Returns the current mean inside of a RunningMean object.
ChemometricsTools.Mean
— Method.Mean(rv::RunningVar)
Returns the current mean inside of a RunningVar object.
ChemometricsTools.MultiNorm
— Method.MultiNorm(T)
Computes the equivalent of the Froebinius norm on a tensor T
. Returns a scalar.
ChemometricsTools.MultiPCA
— Method.MultiPCA(X; Factors = 2)
Performs multiway PCA aka Higher Order SVD aka Tucker, etc. The number of factors decomposed can be a scalar(repeated across all modes) or a vector/tuple for each mode.
Returns a tuple of (Core Tensor, Basis Tensors)
ChemometricsTools.MulticlassStats
— Method.MulticlassStats(Y, GT, schema; Microaverage = true)
Calculates many essential classification statistics based on predicted values Y
, and ground truth values GT
, using the encoding schema
. Returns a tuple whose first entry is a dictionary of averaged statistics, and whose second entry is a dictionary of the form "Class" => Statistics Dictionary ...
ChemometricsTools.MulticlassThreshold
— Method.MulticlassThreshold(yhat; level = 0.5)
Effectively does the same thing as Threshold() but per-row across columns.
Warning this function can allow for no class assignments. HighestVote is preferred
ChemometricsTools.Mutate
— Function.Mutate( L::BinaryLifeform, amount = 0.05 )
Assesses each element in the gene vector (inside of L
). If a randomly drawn value has a binomial probability of amount
the element is mutated.
ChemometricsTools.OneHotOdds
— Method.OneHotOdds(Y)
Calculates the odds of a one-hot formatted probability matrix. Returns a tuple.
ChemometricsTools.PCA_NIPALS
— Method.PCA_NIPALS(X; Factors = minimum(size(X)) - 1, tolerance = 1e-7, maxiters = 200)
Compute's a PCA from x
using the NIPALS algorithm with a user specified number of latent variables(Factors
). The tolerance is the minimum change in the F norm before ceasing execution. Returns a PCA object.
ChemometricsTools.PSO
— Method.PSO(fn, Bounds, VelRange, Particles; tolerance = 1e-6, maxiters = 1000, InertialDecay = 0.5, PersonalWeight = 0.5, GlobalWeight = 0.5, InternalParams = nothing)
Minimizes function fn
with-in the user specified Bounds
via a Particle Swarm Optimizer. The particle velocities are limitted to the VelRange
. The number of particles are defined by the Particles
parameter.
Returns a Tuple of the following form: ( GlobalBestPos, GlobalBestScore, P ) Where P is an array of the particles used in the optimization.
*Note: if the optimization function requires an additional constant parameter, please pass that parameter to InternalParams. This will only work if the optimized parameter(o) and constant parameter(c) for the function of interest has the following format: F(o,c) *
Kennedy, J.; Eberhart, R. (1995). Particle Swarm Optimization. Proceedings of IEEE International Conference on Neural Networks. IV. pp. 1942–1948. doi:10.1109/ICNN.1995.488968
PearsonCorrelationCoefficient( y, yhat )
Calculates The Pearson Correlation Coefficient from vectors Y
and YHat
ChemometricsTools.PercentRMSE
— Method.PercentRMSE( y, yhat )
Calculates Percent Root Mean Squared Error from vectors Y
and YHat
ChemometricsTools.PerfectSmoother
— Method.PerfectSmoother(X; lambda = 100)
Applies an assymetric least squares smoothing function to a a 2-Array X
. The lambda
parameter controls the smoothness. See the reference below for more information.
Paul H. C. Eilers. "A Perfect Smoother". Analytical Chemistry, 2003, 75 (14), pp 3631–3636.
ChemometricsTools.Pipeline
— Method.Pipeline( X, FnStack... )
Construct a pipeline object from vector/tuple of Transforms
. The Transforms vector are effectively a vector of functions which transform data.
ChemometricsTools.Pipeline
— Method.Pipeline(Transforms)
Constructs a transformation pipeline from vector/tuple of Transforms
. The Transforms vector are effectively a vector of functions which transform data.
ChemometricsTools.PipelineInPlace
— Method.PipelineInPlace( X, FnStack...)
Construct a pipeline object from vector/tuple of Transforms
. The Transforms vector are effectively a vector of functions which transform data. This function makes "inplace" changes to the Array X
as though it has been sent through the pipeline. This is more efficient if memory is a concern, but can irreversibly transform data in memory depending on the transforms in the pipeline.
ChemometricsTools.RAFFT
— Method.RAFFT(raw, reference; maxlags::Int = 500, lookahead::Int = 1, minlength::Int = 20, mincorr::Float64 = 0.05)
RAFFT corrects shifts in the raw
spectral bands to be similar to those in a given reference
spectra through the use of "recursive alignment by FFT". It returns an array of corrected spectra/chromatograms. The number of maximum lags can be specified, the lookahead
parameter ensures that additional recursive executions are performed so the first solution found is not preemptively accepted, the minimum segment length(minlength
) can also be specified if FWHM are estimable, and the minimum cross correlation(mincorr
) for a match can dictate whether peaks were found to align or not.
Note This method works best with flat baselines because it repeats last known values when padding aligned spectra. It is highly efficient, and in my tests does a good job, but other methods definitely exist. Let me know if other peak Alignment methods are important for your work-flow, I'll see if I can implement them.
Application of Fast Fourier Transform Cross-Correlation for the Alignment of Large Chromatographic and Spectral Datasets Jason W. H. Wong, Caterina Durante, and, Hugh M. Cartwright. Analytical Chemistry 2005 77 (17), 5655-5661
ChemometricsTools.RMSE
— Method.RMSE( y, yhat )
Calculates Root Mean Squared Error from vectors Y
and YHat
ChemometricsTools.RSquare
— Method.RSquare( y, yhat )
Calculates R^2 from Y
and YHat
ChemometricsTools.Remove!
— Method.Remove!(RM::RunningMean, x)
Removes an observation(x
) from a RunningMean object(RM
) and reculates the mean in place.
ChemometricsTools.Remove
— Method.Remove!(RM::RunningMean, x)
Removes an observation(x
) from a RunningMean object(RM
) and recuturns the new RunningMean object.
ChemometricsTools.SSE
— Method.SSE( y, yhat )
Calculates Sum of Squared Errors from vectors Y
and YHat
ChemometricsTools.SSReg
— Method.SSReg( y, yhat )
Calculates Sum of Squared Deviations due to Regression from vectors Y
and YHat
ChemometricsTools.SSRes
— Method.SSRes( y, yhat )
Calculates Sum of Squared Residuals from vectors Y
and YHat
ChemometricsTools.SSTotal
— Method.SSTotal( y, yhat )
Calculates Total Sum of Squared Deviations from vectors Y
and YHat
ChemometricsTools.SampleSkewness
— Method.SampleSkewness(X)
returns a measure of skewness for vector X
that is corrected for a sample of the population.
Joanes, D. N., and C. A. Gill. 1998. “Comparing Measures of Sample Skewness and Kurtosis”. The Statistician 47(1): 183–189.
ChemometricsTools.SavitzkyGolay
— Method.SavitzkyGolay(X, Delta, PolyOrder, windowsize)
Performs SavitskyGolay smoothing across every row in an Array X
. The window size
is the size of the convolution filter, PolyOrder
is the order of the polynomial, and Delta
is the order of the derivative.
Savitzky, A.; Golay, M.J.E. (1964). "Smoothing and Differentiation of Data by Simplified Least Squares Procedures". Analytical Chemistry. 36 (8): 1627–39. doi:10.1021/ac60214a047.
ChemometricsTools.Scale1Norm
— Method.Scale1Norm(X)
Scales the columns of X
by the 1-Norm of each row. Returns the scaled array.
ChemometricsTools.Scale2Norm
— Method.Scale2Norm(X)
Scales the columns of X
by the 2-Norm of each row. Returns the scaled array.
ChemometricsTools.ScaleInfNorm
— Method.ScaleInfNorm(X)
Scales the columns of X
by the Inf-Norm of each row. Returns the scaled array.
ChemometricsTools.ScaleMinMax
— Method.ScaleMinMax(X)
Scales the columns of X
by the Min and Max of each row such that no observation is greater than 1 or less than zero. Returns the scaled array.
ChemometricsTools.SecondDerivative
— Method.FirstDerivative(X)
Uses the finite difference method to compute the second derivative for every row in X
. Note: This operation results in the loss of two columns.
ChemometricsTools.Shuffle!
— Method.Shuffle!( X, Y )
Shuffles the rows of the X
and Y
data without replacement in place. In place, means that this function alters the order of the data in memory and this function does not return anything.
ChemometricsTools.Shuffle
— Method.Shuffle( X, Y )
Shuffles the rows of the X
and Y
data without replacement. It returns a 2-Tuple of the shuffled set.
ChemometricsTools.SinglePointCrossOver
— Method.SinglePointCrossOver( L1::BinaryLifeform, L2::BinaryLifeform )
Creates two offspring (new BinaryLifeForms) by mixing the genes from L1
and L2
after a random position in the vector.
ChemometricsTools.Skewness
— Method.Skewness(X)
returns a measure of skewness for a population vector X
.
Joanes, D. N., and C. A. Gill. 1998. “Comparing Measures of Sample Skewness and Kurtosis”. The Statistician 47(1): 183–189.
ChemometricsTools.SplitByProportion
— Function.SplitByProportion(X::Array, Y::Array,Proportion::Float64 = 0.5)
Splits an X
and Associated Y
Array along the observations dimension into a 2-Tuple of 2-Tuples based on the Proportion
. The form of the output is the following: ( (X1, Y1), (X2, Y2) )
ChemometricsTools.SplitByProportion
— Function.SplitByProportion(X::Array, Proportion::Float64 = 0.5)
Splits X
Array along the observations dimension into a 2-Tuple based on the Proportion
. The form of the output is the following: ( X1, X2 )
ChemometricsTools.StandardNormalVariate
— Method.StandardNormalVariate(X)
Scales the columns of X
by the mean and standard deviation of each row. Returns the scaled array.
ChemometricsTools.StatsDictToDataFrame
— Method.StatsDictToDataFrame(DictOfStats, schema)
Converts a dictionary of statistics which is returned from MulticlassStats
into a labelled dataframe. This is an intermediate step for automated report generation.
ChemometricsTools.StatsFromTFPN
— Method.StatsFromTFPN(TP, TN, FP, FN)
Calculates many essential classification statistics based on the numbers of True Positive(TP
), True Negative(TN
), False Positive(FP
), and False Negative(FN
) examples.
ChemometricsTools.Threshold
— Method.Threshold(yhat; level = 0.5)
For a binary vector yhat
this decides if the label is a 0 or a 1 based on it's value relative to a threshold level
.
ChemometricsTools.Update!
— Method.Update!(RM::RunningMean, x)
Adds new observation(x
) to a RunningMean object(RM
) in place.
ChemometricsTools.Update!
— Method.Update!(RV::RunningVar, x)
Adds new observation(x
) to a RunningVar object(RV
) and updates it in place.
ChemometricsTools.Update
— Method.Update!(RM::RunningMean, x)
Adds new observation(x
) to a RunningMean object(RM
) and returns the new object.
ChemometricsTools.Variance
— Method.Variance(P::ewma)
This function returns the EWMA control variance.
ChemometricsTools.Variance
— Method.Variance(rv::RunningVar)
Returns the current variance inside of a RunningVar object.
ChemometricsTools.VenetianBlinds
— Method.VenetianBlinds(X,Y)
Splits an X
and associated Y
Array along the observation dimension into a 2-Tuple of 2-Tuples based on the whether it is even or odd. The form of the output is the following: ( (X1,Y1), (X2, Y2) )
ChemometricsTools.VenetianBlinds
— Method.VenetianBlinds(X)
Splits an X
Array along the observations dimension into a 2-Tuple of 2-Tuples based on the whether it is even or odd. The form of the output is the following: ( X1, X2 )
ChemometricsTools.boxcar
— Method.boxcar(X; windowsize = 3, fn = mean)
Applies a boxcar function (fn
) to each window of size windowsize
to every row in X
. Note: the function provided must support a dims argument/parameter.
ChemometricsTools.entropy
— Method.entropy(v)
Calculates the Shannon-Entropy of a probability vector v
. Returns a scalar. A common gain function used in tree methods.
ChemometricsTools.findpeaks
— Method.findpeaks( vY; m = 3)
Finds the indices of peaks in a vector vY with a window span of 2m
. Original R function by Stas_G:(https://stats.stackexchange.com/questions/22974/how-to-find-local-peaks-valleys-in-a-series-of-data) This version is based on a C++ variant by me.
ChemometricsTools.gini
— Method.gini(p)
Calculates the GINI coefficient of a probability vector p
. Returns a scalar. A common gain function used in tree methods.
ChemometricsTools.offsetToZero
— Method.offsetToZero(X)
Ensures that no observation(row) of Array X
is less than zero, by ensuring the minimum value of each row is zero.
ChemometricsTools.plotchem
— Method.plotchem(QQ::{QQ, BlandAltman}; title )
returns either a QQ Plot or a Bland-Altman plot with the defined title
ChemometricsTools.rbinomial
— Method.rbinomial( p, size... )
Makes an N-dimensional array of size(s) size
with a probability of being a 1 over a 0 of 1 p
.
ChemometricsTools.sigmoid
— Method.sigmoid(x)
Applies the sigmoid function to a scalar value X. Returns a scalar. Can be broad-casted over an Array.
ChemometricsTools.ssd
— Method.ssd(p)
Calculates the sum squared deviations from a decision tree split. Accepts a vector of values, and the mean of that vector. Returns a scalar. A common gain function used in tree methods.
(DSX::DirectStandardizationXform)(X; Factors = length(DSX.pca.Values))
Applies a the transform from a learned direct standardization object DSX
to new data X
.
ChemometricsTools.ELM
— Method.(M::ELM)(X)
Makes an inference from X
using a ELM object.
ChemometricsTools.KRR
— Method.(M::KRR)(X)
Makes an inference from X
using a KRR object which wraps a ClassicLeastSquares object.
ChemometricsTools.ewma
— Method.EWMA(P::ewma)(New; train = true)
Provides an EWMA score for a New
scalar value. If train == true
the model is updated to include this new value.
ChemometricsTools.pipeline
— Method.(P::pipeline)(X; inverse = false)
Applies the stored transformations in a pipeline object P
to data in X. The inverse flag can allow for the transformations to be reversed provided they are invertible functions.
ChemometricsTools.ChangeCenter
— Method.ChangeCenter(P::ewma, new::Float64)
This is a convenience function to update the center of a P
EWMA model, to a new
scalar value.