

Section 4: C Debugging, Testing, Formatting

In the previous discussion section, you learned how to use C build and test frameworks to help automate the process of compiling and verifying your programs. In this discussion, we will continue to learn about new tools that can help us better debug, test, and format our programs.

1. Logging Into ecelinux with VS Code

Follow the same process as in the last section. Find a free workstation and log into the workstation using your NetID and standard NetID password. Then complete the following steps (described in more detail in the last section):

• Start VS Code
• Use View > Command Palette to execute Remote-SSH: Connect Current Window to Host...
• Enter netid@ecelinux.ece.cornell.edu
• Use View > Explorer to open folder on ecelinux
• Use View > Terminal to open terminal on ecelinux

For this discussion section you will need to make a copy of the repo we will be using so you can push to it. Go to the section's repo here:

• https://github.com/cornell-ece2400/ece2400-sec04

Click on the "Fork" button. Wait a few seconds and then visit the new copy of this repo in your own person GitHub workspace:

• https://github.com/githubid/ece2400-sec04

Where githubid is your GitHubID. Now clone the GitHub repo we will be using in this section using the following commands:

% source setup-ece2400.sh
% mkdir -p ${HOME}/ece2400
% cd ${HOME}/ece2400
% git clone git@github.com:githubid/ece2400-sec04 sec04
% cd sec04
% tree

Again, where githubid is your GitHubID. The directory includes the following files:

• ece2400-stdlib.h : ECE 2400 course standard library header
• ece2400-stdlib.c : ECE 2400 course standard library implementation
• avg-test.c: source and test for avg function
• sort-test.c: source and test for sort function
• .github/workflows/actions.yml : GitHub Actions configuration script

2. Using GDB for Debugging

There are two kinds of C/C++ programmers in the world: printf debuggers and GDB debuggers. Prof. Batten used to be a printf debuggers but teaching this course has converted him to be a GDB advocate. He will share his perspectives on this in the discussion section.

Let's start by compiling a single-file program that uses a single test case and the ECE2400_CHECK macros to test our ubiquitous avg function.

% cd ${HOME}/ece2400/sec04
% gcc -Wall -g -o avg-test avg-test.c ece2400-stdlib.c
% ./avg-test

Notice how we include the -g option to turn on support for debugging. This code has a bug and should fail the test. Let's start by using printf debugging. Add some extra printfs to observe the state of the program as it executes.

int avg( int x, int y )
{
  printf( "x = %d, y = %d\n", x, y );
  int sum = x + x;
  printf( "sum = %d\n", sum );
  return sum / 2;
}

You should be able to see that the value for the sum variable is incorrect, but the value for the x and y variables are correct. This means we can narrow our focus to line 4 in the above code snippet. Hopefully, you should be able to spot the bug. Fix the bug, recompile, and rerun the program.

Let's now try tracing the execution of this program using GDB. First, remove the extra printfs, undo your bug fix, and then recompile. Then you can start GDB like this

% cd ${HOME}/ece2400/sec04
% gdb -tui avg-test

GDB will drop you into a GDB "prompt" which you can use to interactively execute your program. Your source code will show up at the top, and the GDB prompt is at the bottom. Here are some useful GDB commands:

• break location : set a breakpoint
• run : start running the program
• record : start recording the execution for reverse debugging
• step : execute the next C statement, step into a function call
• next : execute the next C statement, do not step into a function call
• rs : reserve step, undo the execution of current C statement
• print var : print a C variable
• continue : continue on to the next breakpoint
• quit : exit GDB
• refresh : refresh the source code display

GDB is very sophisticated so of course there are many more commands you can use, but these are enough to get started. Let's start by just running the program in GDB:

(gdb) run

Now let's try again, but first let's set a breakpoint to tell GDB to stop at a certain function or line in the program. The following will set a breakpoint at the beginning of the main function.

(gdb) break main

You can see a little b+ marker in the margin next to the first statement in the main function indicating the location of the breakpoint. You might need to use refresh to get GDB to refresh the source code display. We can now use run to start the program running. The execution should stop at the beginning of the function main. You should see the first line of the function highlighted.

(gdb) run > temp

We are using > temp so that any output from printf goes to a file and does not mess up the source code display. We can use record to turn on recording to enable reverse debugging and then we can step through the execution of each C statement using the step command.

(gdb) record
(gdb) step

Keep using step until you get into the avg function You can print out the value of any variable using the print command:

(gdb) print x
(gdb) print y
(gdb) print sum

You can also step backwards using the rs command:

(gdb) rs

Try stepping forward and backward through the avg function and print out various variables to see how they change during the execution. You can use quit to exit.

(gdb) quit

Now fix the bug and rerun the test.

3. Using GCOV for Code Coverage

One you have developed your implementation and the corresponding directed and random tests, you can then move on to understanding the quality of your current set of tests. One way to do this is to use code coverage tools. The idea here is to use a tool which will count how many times each line of code in your program is executed when running all of your tests. If there are lines of code which have never executed, then this is a good indicator that you need more tests to verify that part of your code.

Let's start by recompiling our avg-test.c and turning on code coverage support.

% cd ${HOME}/ece2400/sec04
% gcc -Wall -g --coverage -o avg-test avg-test.c ece2400-stdlib.c
% ./avg-test

This will generate additional data in avg-test.gcda. To make this data easy to read and understand we need to run two more tools: lcov and genhtml like this:

% cd ${HOME}/ece2400/sec04
% lcov --capture --directory . --output-file coverage.info
% lcov --list coverage.info
Reading tracefile coverage.info
                        |Lines       |Functions  |Branches
Filename                |Rate     Num|Rate    Num|Rate     Num
==============================================================
[/home/cb535/vc/git-hub/cornell-ece2400/ece2400-sec04/]
avg-test.c              | 100%     12| 100%     3|    -      0
ece2400-stdlib.c        |20.4%     54|28.6%     7|    -      0
==============================================================
                  Total:|34.8%     66|50.0%    10|    -      0

The tools produce coverage information on each file in the project. You do not need to worry about coverage in the ece2400-stdlib.c file. We can we see that we achieve 100% code coverage in our avg-test.c file. Note that 100% code coverage is not the same as 100% path coverage; ask the instructors for more on this. Also note that 100% code coverage does not mean your program is guaranteed to be correct!

If your code coverage is lower than expected you can drill down and see the coverage statistics for every line in your program by converting the coverage data into a set of HTML reports like this:

% cd ${HOME}/ece2400/sec04
% genhtml coverage.info --output-directory coverage-html
% elinks coverage-html/index.html

You should be able to use your mouse to browse to the report for the avg-test.c source file and verify that every line is being executed in our test.

4. Using clang-format for Autoformatting

The course coding conventions are located here:

• https://cornell-ece2400.github.io/ece2400-docs/ece2400-coding-conventions

Following these conventions and indeed any coding conventions can be quite tedious. Many software companies and open-source software projects use code autoformatters to automate the process of formatting their code according to a set of rules. One such tool is called clang-format. This tool takes a style file that specifies the coding conventions and then tries to reformat your code so it adheres to the style file. We have provided you a style file named .clang-format that adheres to the course coding conventions. You can run clang-format like this;

% cd ${HOME}/ece2400/sec04
% clang-format -style=file avg-test.c

clang-format will output the autoformatted source code. To really see this in action though we need to write some poorly formatted code! Modify the avg function in avg-test.c to look like this:

int avg( int x, int y) {
  int sum = x + y;
    return sum/2; }

Notice how the above code does not following our coding conventions. The space before/after parenthesis is not consistent and the curly braces are on the wrong lines. Run clang-format again like this:

% cd ${HOME}/ece2400/sec04
% clang-format -style=file avg-test.c

Verify that the code is beautiful again. Note that we are not actually modifying the code, just outputting the autoformatted code to the console. If you cat the source code it still is not formatted correctly:

% cd ${HOME}/ece2400/sec04
% cat avg-test.c

You can use the -i command line option to autoformat the code "in-place".

% cd ${HOME}/ece2400/sec04
% clang-format -i -style=file avg-test.c
% cat avg-test.c

With autoformatting students no longer have any excuse for poorly formatted code! However, do keep in mind that the autoformatter does not fix everything. Students will still need to ensure:

• variable and function names following coding conventions
• no irrelevant instructor supplied comments are included
• an appropriate amount of comments are included (not too little, not too much!)
• no generated directories, binaries, etc are committed to their repo

5. Using GitHub Actions for Continuous Integration

Continuous integration is the process of continually integrating, testing, and evaluating your code. We will be using GitHub Actions to facilitate continuous integration. GitHub Actions will automatically run all tests for a project every time code is pushed to GitHub.

To start, you need to enable GitHub Actions for the remote repository on GitHub. Go to this page:

• https://github.com/githubid/ece2400-sec04/actions

Where githubid is your GitHubID. Click on "I understand my workflows, go ahead and enable them". GitHub Actions will report that there are no workflow runs for this repository yet. GitHub Actions looks for a special workflow file in the .github/workflows directory of your repository to determine how to build and test your project. We have already created one of those files for you, and you can see it here:

% cd ${HOME}/ece2400/sec04
% cat .github/workflows/actions.yml

Although it is not important to understand the details of this YAML file, you can still see that it includes comments to first compile avg-test.c and run avg-test:

gcc -g -coverage -o avg-test avg-test.c ece2400-stdlib.c
./avg-test

and then commands to generate and display coverage statistics:

lcov --capture --directory . --output-file coverage.info
lcov --list coverage.info

Let's start by reintroducing the original bug in the avg function, so modify avg-test.c like this:

int avg( int x, int y )
{
  int sum = x + x;
  return sum / 2;
}

Then go ahead and commit all of the work you have done in this tutorial, then push your local commits to the remote repository on GitHub. As a reminder these are the appropriate git commands:

% cd ${HOME}/ece2400/sec04
% git status
% git commit -a -m "working on sec04"
% git push

Revisit the GitHub Actions page for this repository.

• https://github.com/githubid/ece2400-sec04/actions

Click on the current commit "working on sec04", then click on SEC04, then watch as GitHub Actions brings up a virtual machine and runs all the steps to compile and test your code. You should see GitHub Actions failing with a red X. If you click on "avg test" you can see the specific failure message. Now go ahead and fix your code on ecelinux, commit and push your fix, and verify your code is now passing the tests on GitHub Actions.

Note, you should always test your code directly on ecelinux first. Do not use GitHub Actions as the primary way to run your tests. GitHub Actions is only for continuous integration testing.

6. Experimenting with Debugging, Testing, Formatting for PA1

Now that we understand some new tools that can help us better debug, test, and format our programs, we will see how we have automated the process of using these tools in the PA build system. You can use the following steps to update your PA repo to make sure you have received the latest code.

% mkdir -p ${HOME}/ece2400
% cd ${HOME}/ece2400/netid
% git pull
% tree

where netid is your NetID. If you receive a warning from git, you might need to commit your local changes before pulling the latest code.

By default when you use CMake it creates a debug build for you which is setup to use the right flags to enable GDB debugging. In the next discussion section we will learn more about evaluation builds. Let's make sure we have a debug build all setup:

% cd ${HOME}/ece2400/netid/pa1-math
% mkdir build
% cd build
% cmake ..

We have also setup CMake with two targets to automate the process of doing code coverage and autoformatting.

Here is an example of running code coverage:

% cd ${HOME}/ece2400/netid/pa1-math/build
% make clean
% make check
% make coverage

Notice how we do a make clean first. This is because code coverage will include coverage across every single run of all tests and evaluation programs. We want to make sure our coverage analysis is starting from a clean slate. Also keep in mind that analyzing code coverage only makes sense once you pass all of your tests.

Here is an example of running autoformatting:

% cd ${HOME}/ece2400/netid/pa1-math/build
% make autoformat

Note that you can only do autoformatting if you have committed all of your work. This enables us to easily see the changes that autoformatting has done using git diff and to also cleanly undo autoformatting in case something goes wrong.

Finally, all of your PA repos are already setup with a GitHub Actions file:

% cd ${HOME}/ece2400/netid
% cat .github/workflows/run_tests.yml

If you look closely, you should be able to see that the workflow file will run make check and make coverage. Go ahead and check the status of your latest GitHub Actions run here:

• https://github.com/cornell-ece2400/netid/actions

7. To-Do On Your Own

If you have time, try compiling sort-test.c in the section 4 repository and executing the resulting binary. The test should fail. Use GDB debugging to find the bug and fix it. Think critically about whether or not the sort function should swap the given inputs, and then use GDB to trace and see if it is or is not swapping as expected.

After fixing the bug, compile sort-test.c and execute the resulting binary with support for code coverage. Use the code coverage reports to verify that the test has less than 100% code coverage. Add another test case to improve the code coverage to 100%.

After analyzing code coverage, run clang-format with the in-place command line option (-i) to fix up the formatting.

Finally, update the .github/workflows/actions.yml file to compile sort-test.c and run sort-test. You can add this after the avg test block and before the coverage block:

- name: sort test
  run: |-
    gcc -g -coverage -o sort-test sort-test.c ece2400-stdlib.c
    ./sort-test

Then coming all of your changes, push to GitHub, and confirm that GitHub actions is now passing both avg-test and sort-test.