This notebooks compares the 3 modes (:Fast, :Intermediate and :Full) for both Float64 and Float32 reference data depths.
You can run the notebook in either single thread or multi-thread environments. The algorithms will take advantage of multiple cores.
using NeXLSpectrum using DataDeps using Unitful Threads.nthreads()
12
I'll use DataDeps to pull down some data from the NIST website.
register(DataDep("MnNodule", """ Dataset: Deep sea manganese nodule electron excited X-ray microanalysis hyperspectral data set Author: Nicholas W. M. Ritchie (NIST) License: Public Domain Website: https://data.nist.gov/od/id/mds2-2467 Notice: This file is over 600 Mb """, "https://data.nist.gov/od/ds/mds2-2467/MnNodule.tar.gz", "5b5b6623b8f4daca3ff3073708442ac5702ff690aa12668659875ec5642b458d", post_fetch_method = unpack )) register(DataDep("MnNodule_Standards", """ Dataset: Standard spectra for the deep sea manganese nodule electron excited X-ray microanalysis hyperspectral data set Author: Nicholas W. M. Ritchie (NIST) License: Public Domain Website: https://data.nist.gov/od/id/mds2-2467 """, "https://data.nist.gov/od/ds/mds2-2467/MnNodule_Standards.tar.gz", "69283ba72146932ba451e679cf02fbd6b350f96f6d012d50f589ed9dd2e35f1a", post_fetch_method = unpack )) ENV["DATADEPS_ALWAYS_ACCEPT"] = true;
lt = 0.72*4.0*18.0*3600.0/(1024*1024) # 18.0 hours on 4 detectors hs = NeXLSpectrum.compress(HyperSpectrum( LinearEnergyScale(0.0,10.0), Dict{Symbol,Any}( :TakeOffAngle => deg2rad(35.0), :ProbeCurrent => 1.0, :LiveTime => lt, :BeamEnergy => 20.0e3, :Name => "Mn Nodule" ), readrplraw(joinpath(datadep"MnNodule","map[15]")), fov = [ 4.096u"mm", 4.096u"mm"], offset= [ 0.0u"mm", 0.0u"mm" ] ))
1024 × 1024 HyperSpectrum{UInt16,(:Y, :X)}[Mn Nodule), 0.0 + 10.0⋅ch eV, 2048 ch]
refpath = datadep"MnNodule_Standards" refs64 = references( [ reference(n"C", joinpath(refpath, "C std.msa") ), reference(n"Ag", joinpath(refpath, "Ag std.msa") ), reference(n"Al", joinpath(refpath, "Al std.msa") ), reference(n"C", joinpath(refpath, "C std.msa") ), reference(n"Ca", joinpath(refpath, "CaF2 std.msa") ), reference(n"Ce", joinpath(refpath, "CeO2 std.msa") ), reference(n"Cl", joinpath(refpath, "NaCl std.msa") ), reference(n"Cr", joinpath(refpath, "Cr std.msa") ), reference(n"Cu", joinpath(refpath, "Cu std.msa") ), reference(n"Fe", joinpath(refpath, "Fe std.msa") ), reference(n"S", joinpath(refpath, "FeS2 std.msa") ), reference(n"P", joinpath(refpath, "GaP std.msa") ), reference(n"K", joinpath(refpath, "KBr std.msa") ), reference(n"Mg", joinpath(refpath, "Mg std.msa") ), reference(n"O", joinpath(refpath, "MgO std.msa") ), reference(n"Mn", joinpath(refpath, "Mn std.msa") ), reference(n"Na", joinpath(refpath, "NaCl std.msa") ), reference(n"Ni", joinpath(refpath, "Ni std.msa") ), reference(n"Si", joinpath(refpath, "Si std.msa") ), reference(n"Ti", joinpath(refpath, "Ti std.msa") ), reference(n"Zn", joinpath(refpath, "Zn std.msa") ) ], 132.0 ) refs32 = references( [ reference(n"C", joinpath(refpath, "C std.msa") ), reference(n"Ag", joinpath(refpath, "Ag std.msa") ), reference(n"Al", joinpath(refpath, "Al std.msa") ), reference(n"C", joinpath(refpath, "C std.msa") ), reference(n"Ca", joinpath(refpath, "CaF2 std.msa") ), reference(n"Ce", joinpath(refpath, "CeO2 std.msa") ), reference(n"Cl", joinpath(refpath, "NaCl std.msa") ), reference(n"Cr", joinpath(refpath, "Cr std.msa") ), reference(n"Cu", joinpath(refpath, "Cu std.msa") ), reference(n"Fe", joinpath(refpath, "Fe std.msa") ), reference(n"S", joinpath(refpath, "FeS2 std.msa") ), reference(n"P", joinpath(refpath, "GaP std.msa") ), reference(n"K", joinpath(refpath, "KBr std.msa") ), reference(n"Mg", joinpath(refpath, "Mg std.msa") ), reference(n"O", joinpath(refpath, "MgO std.msa") ), reference(n"Mn", joinpath(refpath, "Mn std.msa") ), reference(n"Na", joinpath(refpath, "NaCl std.msa") ), reference(n"Ni", joinpath(refpath, "Ni std.msa") ), reference(n"Si", joinpath(refpath, "Si std.msa") ), reference(n"Ti", joinpath(refpath, "Ti std.msa") ), reference(n"Zn", joinpath(refpath, "Zn std.msa") ) ], 132.0, ftype=Float32 );
Note: I use @time instead of @btime because the calculations a lot of time and in the big picture compilation is a minor contributor.
hsc, hsf = hs[1:4,1:4], hs[1:2:1024,1:2:1024]
(4 × 4 HyperSpectrum{UInt16,(:Y, :X)}[Mn Nodule[(1:4, 1:4)], 0.0 + 10.0⋅ch
eV, 2048 ch], 512 × 512 HyperSpectrum{UInt16,(:Y, :X)}[Mn Nodule[(1:2:1023,
1:2:1023)], 0.0 + 10.0⋅ch eV, 2048 ch])
fit_spectrum(hsc, refs64, mode=:Fast) @time fit_spectrum(hsf, refs64, mode=:Fast);
11.408587 seconds (2.12 M allocations: 4.720 GiB, 2.40% gc time)
fit_spectrum(hsc, refs32, mode=:Fast) @time fit_spectrum(hsf, refs32, mode=:Fast);
9.132086 seconds (2.37 M allocations: 2.400 GiB, 0.75% gc time)
fit_spectrum(hsc, refs64, mode=:Intermediate) @time fit_spectrum(hsf, refs64, mode=:Intermediate);
334.388389 seconds (13.11 M allocations: 364.354 GiB, 7.18% gc time)
fit_spectrum(hsc, refs32, mode=:Intermediate) @time fit_spectrum(hsf, refs32, mode=:Intermediate);
278.343458 seconds (12.59 M allocations: 183.709 GiB, 6.86% gc time)
fit_spectrum(hsc, refs64, mode=:Full) @time fit_spectrum(hsf, refs64, mode=:Full);
579.548141 seconds (6.21 G allocations: 863.035 GiB, 10.20% gc time)
fit_spectrum(hsc, refs32, mode=:Full) @time fit_spectrum(hsf, refs32, mode=:Full);
492.173279 seconds (6.18 G allocations: 517.154 GiB, 13.01% gc time, 0.00% compilation time)