The interfacing of measurement instrumentation to small computers for the purpose of online data acquisition has now become standard practice in the modern laboratory for the purposes of performing signal processing and data analysis and storage, using a large number of digital computer-based numerical methods that are used to transform signals into more useful forms, detect and measure peaks, reduce noise, improve the resolution of over-lapping peaks, compensate for instrumental artifacts, test hypotheses, optimize measurement strategies, diagnose measurement difficulties, and decompose complex signals into their component parts. These techniques can often make difficult measurements easier by extracting more information from the available data. Many of these techniques are based on laborious mathematical procedures and/or analog electronics that were not really practical before the advent of computerized instrumentation. It is important to appreciate the abilities,

This essay covers only
basic topics related to one-dimensional time-series signals, not
two-dimensional data such as images. It uses a pragmatic
approach and is limited to mathematics only up to the most
elementary aspects of calculus, statistics, and matrix math. For
the math phobic, you should know that this essay does not dwell
on the math and that it contains more than twice as many figures
as equations. Data processing without math? Not really! Math is
*essential*, just as it is for the technology of cell
phones, GPS, digital photography, the Web, and computer games.
But you can get *started *using these tools without
understanding all the underlying math and software details.
Seeing it work makes it more likely that you'll want to
understand *how *it works. But in the long run, it's not
enough just to know how to operate the software, any more than
knowing how to use a word processor or a MIDI sequencer makes
you a good author or musician.

Why do I title this document "*signal *processing"
rather than "*data *processing"? By "signal" I mean the
continuous *x*,*y* numerical data recorded by
scientific instruments as time-series,
where *x* may be *time* or another quantity like *energy
*or *wavelength, *as in the various forms of
spectroscopy. "Data" is a more general term that includes *categorical
*data as well. In other words, I'm oriented to data
that you would plot in a spreadsheet using the *scatter*
chart type rather than *bar *or *pie *charts.

At the present time, this work does not cover image processing, wavelet transforms, pattern recognition, or factor analysis. For more advanced topics and for a more rigorous treatment of the underlying mathematics, refer to the extensive literature on signal processing and on statistics and chemometrics.

This site had its origin in one of the experiments in a course called "Electronics and Computer Interfacing for Chemists" that I developed and taught at the University of Maryland in the 80's and 90's. The first Web-based version went up in 1995. Subsequently it has been revised and greatly expanded based on feedback from users. It is still a work in progress and, as such, benefits from continued feedback from readers and users.

This tutorial makes
considerable use of *Matlab*, a
high-performance commercial and proprietary numerical computing
environment and "fourth generation" programming language that is
widely used in research (14, 17, 19, 20), and *Octave*, a free
Matlab alternative that runs almost all of the programs and
examples in this tutorial. There is a good reason why this
language is so massively popular in science and engineering; *it's
powerful, fast, and relatively easy to learn*; it comes
with built-in functions for doing data processing tasks like
matrix math, Fourier transforms, convolution and deconvolution,
multilinear regression, and optimization; you can download
thousands of useful user-contributed functions; it can interface
to C, C++, Java, Fortran, and Python; and it's extensible to symbolic
computing and model-based
design for dynamic and embedded
systems.* There are many code examples in this text that
you can Copy and Paste (or drag and drop) into the
Matlab/Octave command line* to run or modify, which is especially
convenient if you can split
your screen between the two. *If you try
to run one of my scripts or functions and it gives you a
"missing function" error, that means either that you have not
yet downloaded that item from my web site or that you have not
placed it in the "path". Look for the missing item here, download it into your path,
and try again.* Type "help path" at the Matlab/Octave
command prompt for help and related commands.

Some of the illustrations were produced on my old
90s-era freeware signal-processing application for Macintosh
OS8, called S.P.E.C.T.R.U.M. (**S**ignal
**P**rocessing for **E**xperimental **C**hemistry **T**eaching
and **R**esearch / **U**niversity of **M**aryland).
This is now an obsolete and retired version.

Most of the techniques covered in this work can
also be performed in spreadsheets
(11, 22, 23) such as *Excel *or *OpenOffice Calc.
*

Octave
and the OpenOffice
Calc (LibreOffice
Calc) spreadsheet program can be downloaded without cost
from their respective web sites.

All of the Matlab/Octave scripts and functions,
the SPECTRUM program, and all of the spreadsheets used here can
all be downloaded from this site
at no cost; they have received extraordinarily
positive feedback from users. If you try to run one of my
scripts or functions and it gives you a "missing function"
error, look for the missing item on functions.html,
download it into your path, and try again.

If you are unfamiliar with Matlab, read these sections about basics and functions and scripts
for a quick start-up. Matlab is not really a general-purpose
programming languages like C++ or Python; rather, it is
specifically suited to numerical methods, matrix manipulation,
plotting of functions and data, implementation of algorithms,
creation of user interfaces, and interfacing with programs
written in other languages - essentially the needs of numerical
computing by scientists and engineers. Matlab is more loosely
typed and less well structured in a formal sense than
other languages, and thus tends to be more favored by scientists
and engineers and less well liked by computer scientists and
professional programmers. To get a basic language like Python up
to the point where Matlab *starts *takes a considerable
effort and familiarity with computer jargon to install add-on
"packages" of functions that Matlab comes with. This is not a
criticism of Python, which is an extremely capable language,
just an observation of different needs for different fields.

There are several versions of Matlab, including
lower-cost student
and home
versions. See https://www.mathworks.com/pricing-licensing.html
for prices and restrictions in their use. There are also
several other good alternatives to MATLAB, in particular Octave,
which is essentially a Matlab clone, but there is also Scilab, FreeMat, Julia,
and Sage
which are somewhat compatible with the MATLAB language and
which illustrate the influence of Matlab in the scientific
computing community. For a discussion of other possibilities,
see
http://www.dspguru.com/dsp/links/matlab-clones.

"People are generally better persuaded by the reasons which they have themselves discovered than by those which have come into the mind of others."

"A computer does not substitute for judgment any more than a pencil substitutes for literacy. But writing without a pencil is no particular advantage."

"

1. Douglas A. Skoog,

2. Gary D. Christian and James E. O'Reilly,

3. Howard V. Malmstadt, Christie G. Enke, and Gary Horlick,

4. Stephen C. Gates and Jordan Becker,

5. Muhammad A. Sharaf, Deborah L Illman, and Bruce R. Kowalski,

6. Peter D. Wentzell and Christopher D. Brown, Signal Processing in Analytical Chemistry, in

7. Constantinos E. Efstathiou, Educational Applets in Analytical Chemistry, Signal Processing, and Chemometrics. (http://www.chem.uoa.gr/Applets/Applet_Index2.htm)

8. A. Felinger, Data Analysis and Signal Processing in Chromatography, Elsevier Science (19 May 1998).

9. Matthias Otto, Chemometrics: Statistics and Computer Application in Analytical Chemistry, Wiley-VCH (March 19, 1999). Some parts viewable in Google Books.

10. Steven W. Smith, The Scientist and Engineer's Guide to Digital Signal Processing. (Downloadable chapter by chapter in PDF format from http://www.dspguide.com/pdfbook.htm). This is a much more general treatment of the topic.

11. Robert de Levie, How to use Excel in Analytical Chemistry and in General Scientific Data Analysis, Cambridge University Press; 1 edition (February 15, 2001),

12. Scott Van Bramer, Statistics for Analytical Chemistry, http://science.widener.edu/svb/stats/stats.html.

13.

14. Educational Matlab GUIs, Georgia Institute of Technology. (http://spfirst.gatech.edu/matlab/)

15. Jan Allebach, Charles Bouman, and Michael Zoltowski, Digital Signal Processing Demonstrations in Matlab, Purdue University (http://www.ecn.purdue.edu/VISE/ee438/demos/Demos.html)

16. Chao Yang , Zengyou He and Weichuan Yu, Comparison of public peak detection algorithms for MALDI mass spectrometry data analysis, http://www.biomedcentral.com/1471-2105/10/4

17. Michalis Vlachos, A practical Time-Series Tutorial with MATLAB.

18. Laurent Duval , Leonardo T. Duarte , Christian Jutten,

19. Nicholas Laude, Christopher Atcherley, and Michael Heien,

20. P. E. S. Wormer, Matlab for Chemists, http://www.math.ru.nl/dictaten/Matlab/matlab_diktaat.pdf

21. Martin van Exter, Noise and Signal Processing, http://molphys.leidenuniv.nl/~exter/SVR/noise.pdf

22. Scott Sinex, Developer's Guide to Excelets, http://academic.pgcc.edu/~ssinex/excelets/

23. R. de Levie, Advanced Excel for scientific data analysis, Oxford University Press, New York (2004)

24. S. K. Mitra, Digital Signal Processing, a computer-based approach, 4th edition, McGraw-Hill, New York, 2011.

25. “Calibration in Continuum-Source AA by Curve Fitting the Transmission Profile” , T. C. O'Haver and J. Kindervater,

26. “Estimation of Atomic Absorption Line Widths in Air-Acetylene Flames by Transmission Profile Modeling”, T. C. O'Haver and Jing-Chyi Chang,

27. “Effect of the Source/Absorber Width Ratio on the Signal-to-Noise Ratio of Dispersive Absorption Spectrometry”, T. C. O'Haver,

28. “Derivative Luminescence Spectrometry”, G. L. Green and T. C. O'Haver,

29. “Derivative Spectroscopy”, T. C. O'Haver and G. L. Green,

30. “Numerical Error Analysis of Derivative Spectroscopy for the Quantitative Analysis of Mixtures”, T. C. O'Haver and G. L. Green,

31. “Derivative Spectroscopy: Theoretical Aspects”, T. C. O'Haver,

32. “Derivative and Wavelength Modulation Spectrometry," T. C. O'Haver,

33. “A Microprocessor-based Signal Processing Module for Analytical Instrumentation”, T. C. O'Haver and A. Smith,

34. “Introduction to Signal Processing in Analytical Chemistry”, T. C. O'Haver,

35. “Applications of Computers and Computer Software in Teaching Analytical Chemistry”, T. C. O'Haver,

36. “The Object is Productivity”, T. C. O'Haver,

37. Analysis software for spectroscopy and mass spectrometry, Spectrum Square Associates ( http://www.spectrumsquare.com/).

38. Fityk, a program for data processing and nonlinear curve fitting. (http://fityk.nieto.pl/)

39. Peak fitting in Origin (http://www.originlab.com/index.aspx?go=Products/Origin/DataAnalysis/PeakAnalysis/PeakFitting)

40. IGOR Pro 6, software for signal processing and peak fitting (http://www.wavemetrics.com/index.html)

41. PeakFIT, automated peak separation analysis, Systat Software Inc..

42. OpenChrom, open source software for chromatography and mass spectrometry. (http://www.openchrom.net/main/content/index.php)

43. W. M. Briggs, Do not smooth times series, you hockey puck!, http://wmbriggs.com/blog/?p=195

44. Nate Silver, The Signal and the Noise: Why So Many Predictions Fail-but Some Don't , Penguin Press, 2012. ISBN 159420411X . A much broader look at "signal" and "noise", aimed at a general audience, but still worth reading.

45. David C. Stone, Dept. of Chemistry, U. of Toronto, Stats Tutorial - Instrumental Analysis and Calibration.

46. Streamlining Digital Signal Processing: A Tricks of the Trade Guidebook, Richard G. Lyons, John Wiley & Sons, 2012.

47. Atomic spectra lines database. http://physics.nist.gov/PhysRefData/ASD/ and http://www.astm.org/Standards/C1301.htm

48. Curve fitting to get overlapping peak areas (http://matlab.cheme.cmu.edu/2012/06/22/curve-fitting-to-get-overlapping-peak-areas)

49. Tony Owen, Fundamentals of Modern UV-Visible Spectroscopy, Agilent Corp, 2000.

50. Nicole K. Keppy, Michael Allen, Understanding Spectral Bandwidth and Resolution in the Regulated Laboratory, Thermo Fisher Scientific Technical Note: 51721. http://www.analiticaweb.com.br/newsletter/02/AN51721_UV.pdf

51. Martha K. Smith, "Common mistakes in using statistics", http://www.ma.utexas.edu/users/mks/statmistakes/TOC.html

52. Jan Verschelde, “Signal Processing in MATLAB”, http://homepages.math.uic.edu/~jan/mcs320s07/matlec7.pdf

53. Howard Mark and Jerome Workman Jr, “Derivatives in Spectroscopy”, Spectroscopy 18 (12). p.106.

54. Jake Blanchard, Comparing Matlab to Excel/VBA, https://blanchard.ep.wisc.edu/PublicMatlab/Excel/Matlab_VBA.pdf

55. Ivan Selesnick, "Least Squares with Examples in Signal Processing", http://eeweb.poly.edu/iselesni/lecture_notes/least_squares/

56. Tom O'Haver, "Is there Productive Life after Retirement?",

57. http://www.dsprelated.com/, the most popular independent internet resource for Digital Signal Processing (DSP) engineers around the world.

58. John Denker, "Uncertainty as Applied to Measurements and Calculations", http://www.av8n.com/physics/uncertainty.htm

59. T. C. O'Haver, Teaching and Learning Chemometrics with Matlab,

60. Allen B. Downey, "Think DSP", Green Tree Press, 2014. (164-page PDF download). Python code instruction using sound as a basis.

61. Purnendu K. Dasgupta, et. al, "Black Box Linearization for Greater Linear Dynamic Range: The Effect of Power Transforms

on the Representation of Data",

62. Joseph Dubrovkin, Mathematical Processing of Spectral Data in Analytical Chemistry: A Guide to Error Analysis, Cambridge Scholars Publishing, 2018, 379 pages. ISBN 978-1-5275-1152-1. Link.

63. Power Law Approach as a Convenient Protocol for Improving Peak Shapes and Recovering Areas from Partially Resolved Peaks, M. Farooq Wahab, Fabrice Gritti, Thomas C. O’Haver, Garrett Hellinghausen, Daniel W. Armstrong,

64. T. C. O’Haver, *Interactive Simulations of
Basic Electronic and Operational Amplifier Circuits*, https://terpconnect.umd.edu/~toh/ElectroSim,
(1996)

65. Signal Processing at Rice University. (http://dsp.rice.edu/software/)

66. Steven Pinker, The Sense of Style: The Thinking
Person's Guide to Writing in the 21st Century, New York, NY: Penguin, 2004.

67. Joseph Dubrovkin, Signal
Processing project on ResearchGate.

68. Separations at the
Speed of Sensors, D. C. Patel, M. Farooq Wahab, T. C. O’Haver,
and Daniel W. Armstrong, Analytical Chemistry **2018 **90
(5), 3349-3356, DOI: 10.1021/acs.analchem.7b04944

69. MF Wahab, TC O'Haver, F. Gritti,
G.Hellinghausen, and DW Armstrong, “Increasing chromatographic
resolution of analytical signals using derivative enhancement
approach,” Talanta, vol. 192, pp. 492–499, **2019**

70. Yuri Kalambet,
"Reconstruction of exponentially modified functions",** 2019**.** **DOI:
10.13140/RG.2.2.12482.84160. Link.

71. Yuri Kalambet, Yuri Kozmin, Andrey
Samokhin, "Comparison of integration rules in the case of very
narrow chromatographic peaks", *Chemometrics and
Intelligent Laboratory Systems* 179**2018**. DOI:
10.1016/j.chemolab.2018.06.001

73. Allen, L. C., Gladney, H. M., Glarum, S. H.,
J. Chem. Phys. 40, 3135 (**1964**).

74. J. W.
Ashley, Charles N. Reilley, "De-Tailing and Sharpening
of Response Peaks in Gas Chromatography", *Anal. Chem.*,
37, 6, 626-630, **1965**.

*Spectrochemica Acta*,
Vol 48B, p. 1393-1409, **1993**.

76. S. Sterlinski, "A Method for Resolution
Enhancement of Interfering Peaks in Ge(Li) Gamma-Ray Spectra", *J.
of Radioanalytical Chemistry*, 31, 195-226, **1****976**.

Updated June, 2019. This page is part of "

Unique visits since May 17, 2008: