Using ChatGPT for Efficient Debugging
It's hard to deny that Large Language Models (LLMs) are making a profound impact on various industries and applications, revolutionizing the way we work and interact. Even though the initial hype around ChatGPT has calmed down since its release about six months ago (in November 2022), its influence remains significant. It seems that autoregressive LLMs will continue to be a part of our lives in the near future, and it is worth developing skills to interact with them, both as a developer and a user.
As Chip Huyen stated in her blog post, it is relatively easy to achieve something impressive with LLMs, but it is quite challenging to build something production-ready considering the limitations and potential issues that LLMs currently have. However, while the research and engineering community is actively working to address these challenges, it is worth acknowledging the fact that individuals can already benefit greatly from LLMs, at least using them as personal assistants for everyday non-critical tasks or as collaborators for brainstorming.
In my previous article, I discussed the best practices of prompt engineering, providing insights to help you develop local LLMs-based applications. In this post, I will share a set of techniques that enable you to utilize models such as ChatGPT for effective code Debugging and accelerated learning of programming. We will also take a look at the example prompts for writing and explaining code. These techniques will be valuable not only when interacting with ChatGPT but also when seeking assistance from your colleagues or even tackling programming challenges independently.
This article is primarily targeted toward beginners, so I tried to provide illustrative examples and explanations. I hope these techniques will assist you in understanding and troubleshooting code more efficiently.
General Framework for Code Debugging
In fact, ChatGPT has not made significant changes to the debugging process. The great thing is, now you can easily connect with a virtual colleague without worrying about being a bother or feeling hesitant to ask stupid questions! But the techniques that we will consider exist as long as software engineering exists, and therefore will be useful not only when interacting with LLMs, but also for a better understanding of the process and more effective interaction with coworkers.
To find a bug in your code, you only need two essential steps (there's three actually):
- Isolate the bug and demonstrate it with the minimum amount of code;
- Make an assumption about your error and test it;
- Iterate with more assumptions until you find a fix.
While you can start using ChatGPT right away, it's actually a better idea to begin by reproducing the error for a few reasons. First of all, it might be challenging to include all the related points and explain exactly what you're trying to achieve within the context of the language model. Secondly, it will allow you to gain a better understanding of the issue and possibly find the error yourself. Let's see.
By the way, in this post I am using the vanilla version of ChatGPT (GPT-3.5), but for coding tasks, GPT-4 is typically more proficient.
Step 1: Isolate and Reproduce the Problem with the Minimum Amount of Code
The first step is to reproduce the problem. As we know, the majority of issues can still be resolved with the classic "turn it off and on again". It's possible that you might have become tangled up with the code execution order in Jupyter Notebook.
If possible (and it typically is), it's recommended to write new code that throws the same error and keep it as simple as possible.
Let's consider the example of a TypeError: 'int' object is not iterable, which occurs when you try to iterate over some_integer instead of using the range(some_integer) construct.
Bad example: a function calls another function that then invokes a method of a class. At first glance, it may require some time to determine where the actual computation occurs, despite this being a relatively simple example. Similarly, for models, it becomes more challenging to locate the relevant information among unrelated details.
Better example: get rid of the class by moving the functionality of do_some_work() function (which is causing the error) directly into the function we are calling.
Besides the fact that we still do a terrible job with the variable naming conventions (remember, variable names should be descriptive and meaningful!), this code is still easier to debug and understand.
Even better example: we can get rid of some_function() as well.
Overall, we have shortened the code by more than half. Compare how much easier it is to find a bug in it.
In the context of pandas, for example, this principle can mean not using the original dataframe. Let's consider a situation where we want to calculate the average salary for each position using our data and encounter a KeyError. Here is a bad example:
First, we can't be sure that the dataframe contains the data provided in the comments. Indeed, we only need two columns from it, and if we create a similar mini version, it will be much easier to understand that we simply misspelled the salary column (Salary vs salary).
By the way, ChatGPT is quite good at generating dummy data, so it can be helpful here too!
There are countless types of errors, and it's impossible to list them all, of course. Overall, try to modify the code in such a way that it produces the same error as you encountered, but make it as easy as possible to understand quickly.
Due to the so-called "rubber duck debugging," this step often helps you understand the cause of the problem on your own, without seeking external assistance. For example, if your mini-code doesn't generate the same error, you're already halfway toward finding a solution. However, even if it does, it's still a positive outcome.
