In
2022, the artificial
intelligence research and deployment company OpenAI, introduced
a
conversational “large-language” model called ChatGPT,
which they had trained on billions of pages of text, almost
every book ever
published, all of Wikipedia, and selected websites. (You can try
it for no cost
at https://chat.openai.com/.)
The current version, as of mid-2024, is ChatGPT-4o. Since that
introduction,
there have been many other competing chatbots, such as Google’s Gemini,
Microsoft’s CoPilot,
Perplexity,
and Anthropic’s Claude.
There
are
even AI services that are especially oriented toward code
development, such
as GitHub Copilot,
Amazon CodeWhisperer, Codex, and Tabine.
The
strength of chatbots is
in language interpretation and writing. For example, chatbots
are quite good at
tasks like suggesting possible titles for papers, talks, or
proposals that you
are working on. Just feed it all or a portion of what you have
written. They
can also paraphrase and outline, which can be useful for writing
condensed
summaries or abstracts. Their knowledge base is extraordinarily
wide and they
can often answer very specific questions about technical topics.
For example,
chatbots can offer opinions and suggestions as to whether your
writing is
suitable for a particular audience and will offer to re-write
the whole thing,
for example to make it more formal or less formal. They are
often better than
Google at answering very specific questions about hardware or
software programs
that would otherwise force you to plow through pages and pages
of
documentation.
But AIs can go well
beyond human
language tasks; they can also write code in several computer
languages commonly
used by scientists, such Matlab, Python, Wolfram Mathematica, C/C++,
R, Pascal, Fortran, HTML, JavaScript, Excel macros,
etc.
But the
critical question is: Can AIs write code easier, quicker,
and more
accurately that a human, specifically a human who is a busy
scientist who is
not a trained computer programmer?
Based
on my limited testing,
ChatGPT and Claude can generate working code in Matlab or Python
for quite a
range of signal processing applications, if you describe the
task adequately.
I found that they could write code that works for many
signal processing
tasks, if the description is
sufficiently complete. But for more complex tasks, its code might
not work at all
or might not do what you expect. Most often, the cause of failure
is that I
have written the prompt unclearly or incompletely. It’s somewhat
misleading
that, even in cases where a chatbot’s code does not work, it
is presented in
good style, divided into clearly identified sections,
usually with
explanatory comments for each line, appropriate indentation,
examples of use,
and even warning that the code may fail under certain
circumstances (e.g.,
division by zero, etc.).
A simple
example where chatbots work perfectly is Caruana's Algorithm
(page 172), which
is a fast way to estimate the parameters of a signal peaks that
are locally
Gaussian near their maxima. I asked the chatbots to create a
function that “accepts
two vectors x and y approximating a digitally sampled peak,
takes the natural
log of y, fits a parabola to the resulting data, then computes
the position,
FWHM, and height of the Gaussian from the coefficients of the
parabolic fit”.
In
this case the chatbot does all the required algebra and creates
code that is
functionally identical to my hand-coded version.
Next, I asked both
ChatGPT and Microsoft’s
CoPilot to “Write a Matlab script that creates a plot with
two horizontal
subplots, the top one showing a signal with several peaks and
some random
noise, and the bottom one showing the results of applying a
smooth of three
different widths to that signal, each shown in a different
line style and
color, with a legend identifying each plot and its smooth
width”. This
results in a working
script that generates
the graphic shown here, just as requested, although CoPilot
included only one
of the smooth widths in its legend. (Note that the chatbots are
forced to make
choices for several things that I did not specify exactly,
including the shape
of the peaks, the sampling rate and signal length, and the three
smooth widths.
Moreover, it adds titles to both subplots, even though I did not
specify that
detail). It is particularly handy that, if you want to generate
a Python
equivalent for example, you can simply say “How can I do that
in Python”
and it creates a working python
script that imports the
required libraries and generates an almost
identical graphic. Although in this
case it chose a much lower sampling rate. (The same may be true
of the other
languages that it knows, but I have not tested that). You can
also feed it a
program written in one of its languages and ask it to convert it
into another
of its languages.
Sometimes the code
generated by chatbots will
result in an error, notated by a red warning message in the
console of both
Matlab and Python/Spyder. You can just copy that error message
and paste it
directly into the chatbot and it will attempt to fix the
problem, often
successfully.
Another query that
created well-structured, easily-modified, functional code is “Create
a
signal consisting of three noisy Gaussian peaks, determine
each peak height,
position, and width by iterative curve fitting, repeat the
process 10 times
with the same Gaussian but with different independent samples
of random noise,
and then compute the mean and standard deviation of the peak
heights,
positions, and widths”.
Matlab
code. Python
code.
A more useful
variation is a function that
“fits
the sum of n Gaussian functions to the data in x and y, given
the value of n
and initial estimates of the positions and widths of each
Gaussian, then
repeats the fit 10 times with slightly different values of the
initial
estimates, takes the one with the lowest fitting error and
returns its values
for the position, width, and height of all the best-fit
Gaussians”.
ChatGPT and Claude produce code that works and, without me even
asking, creates
a script that demonstrates it with a nice graph.
Some
other cases where it
worked well: “Create a test function that
accepts a signal
(x,y), then applies the perpendicular drop method to measure
the peak widths.
Plot the test signal and draw the vertical lines at each
minimum.” Both
ChatGPT and Claude handled this neatly, as shoen in the
graphaic.
Other
examples include:
·
creating a Kalman filter,
·
demonstrating the
bootstrap method,
·
detecting peaks and estimating
their areas by
perpendicular drop (shown on the right),
·
demonstrating wavelet denoising
(for which Claude’s
version worked better that ChatGPT’s),
·
and creating a function that
symmetrizes
exponentially broadened peaks by first-derivative addition,
where Claude
worked better than ChatGPT, Gemini, or Perplexity (graphic).
Clearly asking a
chatbot to perform routine
tasks such as these is quick and convenient, especially if you
are creating the
equivalent code in more than one language; it spits out the code
faster than I
can type. For larger more complex projects, you could break up
the code into
smaller chunks or functions that a chatbot can handle separately
and that
you can test separately, then combine them later as
needed.
A
chatbot always
presents its results nicely formatted,
with correct spelling and grammar, which many people interpret
as a “confident
attitude”. This inspires trust in the results, but just as for
people, confident
does not always mean correct. There are several
important caveats:
First,
the code that a
chatbot generates is not necessarily unique; if you ask it to
repeat the task,
you’ll sometimes get different code (unless the task is so
simple that there is
only one possible way to do it correctly). This is not
necessarily a flaw;
there is often more than one way to code a function for a
particular purpose.
If the code requires variables to be defined, the names of those
variables will
be chosen by a chatbot and
won’t always
be the same from trial to trial. Moreover, unless you specify
the name
of the function itself, a chatbot
will
choose that name as well, based on what the function does).
Second,
and more importantly,
there may be more than one way to interpret your request,
unless you
have very carefully worded it to be unambiguous. Take the
example of data
smoothing (page 42). On the face of it, this is a
simple process.
Suppose we ask for a function that performs a sliding average
smooth of width n
and applies it m times. How will that request be
interpreted? If you
simply say “…apply an n-point sliding average smooth to the y
data and
repeats it m times", you will get this code,
which does what you asked but probably not what you want. The
point of applying
repeat smooths is to apply the smooth again to the
previously smoothed data,
not to the original data, as this code does. The general
rule is that n
passes of a m-width smooth results in a center-weighted
smooth of width n*m-n+1.
You will get what you want if you ask ChatGPT for a function
that “applies
an n-point sliding average smooth to y and then repeats the
smooth on the
previously smoothed data m times", a small but critical
difference,
resulting in this code,
whereas the previous code returns a singly-smoothed result no
matter the value
of m.
Third,
there may be
unspecified details or side effects that may require addressing,
such as the
expectation that the number of data points in the signal after
processing should be the same as before processing. In
the case of
smoothing, for example, there is also the question of what to do
about the
first n and last n data points, for which there
are not enough
data points to compute a complete smooth (see page 45). There is also the requirement
that the
smooth operation should be constructed so that it does not shift
the x-axis
positions of signal features, which is critical in many
scientific applications.
Human-coded smooth algorithms, such as fastsmooth,
consider all these details.
Another
example of unspecified
details is
the measurement of the full width at half-maximum (FWHM) of
smooth peaks of any
shape. The function I wrote for that task is “halfwidth.m”.
I used its description as the ChatGPT query: “…a function
that accepts a
time series x, y, containing one or more peaks, and computes
the full width at
half maximum of the of the peak in y nearest xo. If xo is
omitted, it computes
the halfwidth from the maximum y value”. The AI came up
with “computeFWHM.m”,
which works well if the sampling rate is high enough. However,
the AI’s version
fails to interpolate between data points when the half-intensity
points fall between
data points and thus is inaccurate when the sampling rate is
low, because I
did not specify that it should do so. “CompareFWHM.m”
compares both functions on some
synthetic data
with adjustable sampling rate.
Also,
chatbots cannot
evaluate the graphical results of their calculations. For
example, if you ask
it to apply some operation, and the graphical result is clearly
incorrect, it
does not have a clue. But you can complain about that to the
chatbot and it will
attempt a correction, sometimes successfully.
Another
technical kink
relates to the common Matlab practice of saving functions that
you have written
as a separate file in the path and then later calling that saved
function from
a script that you are writing, relying on Matlab to find it in
the path. If you
ask ChatGPT to convert that new script to another language, you
must embed your
external functions directly into the code (Matlab
R2016b or later).
A more challenging task is iterative fitting
of noisy
overlapping peaks (page 202). I asked ChatGPT to “fit
the sum of n
Gaussian functions to the data in x and y, given initial
estimates of the
positions and widths of each Gaussian, returning the position,
width, and
height of all the best-fit Gaussians”. ChatGPT came up
with the
attractively coded “iterativefitGaussians.m”.
The
closest hand-coded equivalent in my tool chest was fitshape2.m (page
202);
both
codes work, are about the same length, and require similar input
and
output arguments. There is seldom a uniquely
correct answer to iterative
fitting problems, but
the difference
in performance between these two codes is instructive. The
self-contained
script “DemoiterativefitGaussians2.m”
compares these two functions for a simulated signal with three
noisy peaks
whose true parameters are set in lines 13-15. For an “easy” test
case, with
little peak overlap, both codes work well. But
if the peaks
overlap significantly, ChatGPT’s code fails to yield a good fit but my
fitshape2.m code
does work. The
difference is probably due to the different minimization
functions employed
(lsqcurvefit.m vs fminsearch.m).
One
signal
processing task that gave the chatbots some trouble was Fourier
self-deconvolution (page 113).
After
several attempts at writing prompts, ChatGPT did the job with
the prompt: “Matlab
function that performs Fourier self-deconvolution as a means
for sharpening the
width of signal peaks in x,y, by computing the Fourier
transform of the
original signal, dividing that by the Fourier transform of a
zero-centered
Gaussian function of width deconvwidth, performing an inverse
Fourier transform
of the result, and applies a smoothing function of width
smoothwidth to the
deconvoluted result”. Function code, test script).
Segmented
processing functions,
treated on page 331,
are very useful in cases where the signal varies in character
along its length
so that a single processing function setting is not optimum.
Basically, the
idea is that you break up the signal into segments, process each
segment with
different settings, then reassemble the processed segments. A
simple example is
segmented smoothing (page 55). The following prompt
seems like a reasonable
explanation of what is required:
“Write
a function that accepts a signal vector y and a vector s
of
length n, makes n copies of y, smooths
each one with a
smooth width equal to the nth element of the
vector s,
then reassembles the smoothed signal from the nth
segment of
the nth copy.”
But when the resulting function is applied to
the signal in
blue on the left, the smoothed signal, in red, exhibits serious
discontinuities. The problem here is caused by the “end effects”
of smoothing,
described on page 45.
The
problem is solved by an
explicit description of a process that will avoid the problem:
“Matlab function
that accepts a signal vector y, a number of segments n, an
initial smooth width
Wstart and a final smooth width of Wend, creates vector S
of length n,
whose first element is Wstart, last element is Wend, and other
elements are
evenly distributed between Wstart and Wend. The function makes n
copies
of y, smooths each one with a smooth width equal to the
nth
element of the vector S, then reassembles the smoothed
signal from the nth
segment of the nth copy”.
For more complex
functions, it
rapidly gets to the point where it’s impossible to completely
describe the
function in enough detail. This is especially true of
open-source code that has
been developed incrementally over time, with lots of added
options and features
suggested by hundreds of users, such as my 5500-line interactive
peak fitter, ipf.m (page 414),
or
my 4000-line peakfit.m
function (page
394) which
takes up to 12
input arguments, provides 8 output values, and fits 50 different
peak shapes. Even
describing such functions completely for a chatbot would
be tedious at
best. It’s unrealistic to expect any chatbot to completely
replace such
efforts.
ChatGPT
cannot
write Matlab Live Scripts or Apps because they are not text
files.
However, you can easily convert a chatbot-generated script into
a Live Script,
as described on page 365,
or you can have chatbots write keyboard-controlled interactive
functions (like
those described on page 394).
I
tested this with a simple pan-zoom-smooth function, using the
prompt: “Write
a Matlab function that accepts and plots a signal as an x,y
pair and has pan
and zoom functions that are interactively controlled by the
keyboard cursor
keys, using the left and right cursor keys to pan back and
forth and the up and
down keys to zoom in and out. Add a smoothing function that
uses the A and Z
keys to increase and decrease the smooth width.”
Oddly, ChatGPT and Claude failed with the zoom part of this
function, for
reasons that are not clear, but Perplexity
succeeded. This code could serve as the bones of a customized
interactive
function with your own additional keypress functions added,
modeled on the
smooth function (with the smooth width controlled by the “a” and
“z” keys).
Much
more thought and
experience go into hand-coded programs than AI generated ones.
An experienced
human coder knows the typical range of applications and
anticipates typical
problems and limitations, especially as those apply to the
specific field of applications,
such as signals generated by different types of scientific
instrumentation. Of
course, AIs do know a great deal about specific computer
languages and their
capabilities and inbuilt functions, which can be very useful,
especially if you
are re-coding an algorithm in a new or less familiar computer
language. But AIs
are no replacement for human experience. It goes without saying
that you must
test any code that a chatbot gives you, just as you must test
your own code.
Another
useful service that
chatbots can perform is to recognize the function and purpose of
parts of a
program that you are trying to understand. For example, if you
feed ChatGPT this
script, sft.m, without any comment, it will
recognize it as a
(slow) Fourier transform calculation. It will also recognize
mathematical
expressions and equations, such as the compound interest
equation V = S*(1 +
R)^T.
It
seems likely that in the
future, AI services such as chatbots will be much more capable
and more widely
available. Development is ongoing, and millions of users are
already using the chatbot
servers with free or subscription-based accounts.
How
to use ChatGPT to code any programming language
Using
ChatGPT as Your Programming Assistant
