A memory leak issue which might be confusing for students struggling with ICS Project (aka, PA) of Nanjing University.
Notification
The issue mentioned in this article has been discovered by the developer community for a while(And probabily been fixed so far). But for freshmen in this field, such as students taking NJU ICS course, this is not well-known and may cause serious confusion. Geng Tiancheng(top-mind) discovered this during his implementing of PA(GitHub issue page). And he delegated me, Wenrui Huang(rijuyuezhu), to write a blog recording the issue, in order to help students who meet the same situation.
Bug Summary
The environment
All the tests are based on the environment listed below.
$ apt show libreadline8 Package: libreadline8 Version: 8.1.2-1 Priority: important Section: libs Source: readline Origin: Ubuntu Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com> Original-Maintainer: Matthias Klose <doko@debian.org> Bugs: https://bugs.launchpad.net/ubuntu/+filebug Installed-Size: 472 kB Depends: readline-common, libc6 (>= 2.33), libtinfo6 (>= 6) Task: minimal, server-minimal Download-Size: 153 kB APT-Manual-Installed: no APT-Sources: https://mirror.nju.edu.cn/ubuntu jammy/main amd64 Packages Description: GNU readline and history libraries, run-time libraries The GNU readline library aids in the consistency of user interface across discrete programs that need to provide a command line interface. . The GNU history library provides a consistent user interface for recalling lines of previously typed input.
Overview
When following the instruction of PA gitbook: PA1 ~ 监视点 ~ sanitizer - 一种底层的assert, (find the page in ICS2023), students turn on the functionality of address sanitizer. This tools help to provide basic examinations of array access out of bounds as well as memory leaks.
However, though this tool helps a lot dealing with memory leak issues caused by the students(for example, non-matching malloc and free), some of the students will meet strange memory leak fault that indicating there is an issue in the lib readline, like the message below:
Direct leak of 32 byte(s) in 1 object(s) allocated from: #0 0x7f9b452b4887 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145 #1 0x7f9b45c5dbac in xmalloc (/lib/x86_64-linux-gnu/libreadline.so.8+0x39bac)
Direct leak of 32 byte(s) in 1 object(s) allocated from: #0 0x7f73104b4887 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145 #1 0x7f7310f5fbac in xmalloc (/lib/x86_64-linux-gnu/libreadline.so.8+0x39bac)
Indirect leak of 402 byte(s) in 21 object(s) allocated from: #0 0x7f73104b4887 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145 #1 0x7f7310f5fbac in xmalloc (/lib/x86_64-linux-gnu/libreadline.so.8+0x39bac)
We learn that this issue is related to the history functionality of readline lib. However, because the shared library is not compiled with flag -g, and gdb also cannot find the position of memory leak, the debug process may be hard, even though we know it is the library readline to blame.
Research the source repo of lib readline
It is great that readline is open-source, and we can easily obtain the source code of the library. Links are as follows:
$ git log --oneline --graph * 7274faa (HEAD -> master, origin/master, origin/HEAD) Readline-8.2 patch 1: fix crash when readline is started with an invalid locale specification * f7a382f (tag: readline-8.2) readline-8.2 distribution sources and documentation * 5263c0d Readline-8.1 patch 2: fix redisplay of some characters > 128 in certain single-byte encodings * 9ba3434 Readline-8.1 patch 1: fix version comparisons in startup files * cf3c762 (tag: readline-8.1) Readline-8.1 distribution sources and documentation * c5ad6be problems restoring the history file are not signaled correctly to the calling application ...
Some developer reports that the bug is fixed in the version readline-8.2, and let us verify that. Switch to the readline-8.2 tag and compile the lib (according to README)
1 2 3
$ git checkout readline-8.2 $ ./configure $ make
The readline lib provides some examples, and we can use that for our bug reproduction.
1 2
$ cd examples $ make
We can use, for example, rltest for test. But to detect memory leak, we need some tools. Valgrind is a very powerful tool for such a need. Run rltest with valgrind:
1 2 3 4 5 6 7
$ valgrind ./rltest ==2341346== Memcheck, a memory error detector ==2341346== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al. ==2341346== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info ==2341346== Command: ./rltest ==2341346== readline$
Remember what to do to reproduce the bug? First input random string and enter <CR>, then input random string and enter up arrow, and finally enter <CR> and quit the program(We can use Ctrl-C here):
$ valgrind ./rltest ==2341346== Memcheck, a memory error detector ==2341346== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al. ==2341346== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info ==2341346== Command: ./rltest ==2341346== readline$ abc abc readline$ abc abc # Note: this string is initially not abc, but turns to abc after entering up arrow ^C ==2341346== ==2341346== <Some other info> ==2341346== ==2341346== LEAK SUMMARY: ==2341346== definitely lost: 0 bytes in 0 blocks ==2341346== indirectly lost: 0 bytes in 0 blocks ==2341346== possibly lost: 0 bytes in 0 blocks ==2341346== still reachable: 222,888 bytes in 271 blocks ==2341346== suppressed: 0 bytes in 0 blocks ==2341346== Rerun with --leak-check=full to see details of leaked memory ==2341346== ==2341346== For lists of detected and suppressed errors, rerun with: -s ==2341346== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
We find that the memory lost is zero, so the bug is fixed in readline-8.2.
And we checkout the former version:
1 2 3
$ git checkout HEAD~ Previous HEAD position was f7a382f readline-8.2 distribution sources and documentation HEAD is now at 5263c0d Readline-8.1 patch 2: fix redisplay of some characters > 128 in certain single-byte encodings
And do the same test as above, we get info like:
1 2 3 4 5 6 7 8 9 10
==2355120== LEAK SUMMARY: ==2355120== definitely lost: 32 bytes in 1 blocks ==2355120== indirectly lost: 66 bytes in 3 blocks ==2355120== possibly lost: 0 bytes in 0 blocks ==2355120== still reachable: 198,179 bytes in 249 blocks ==2355120== suppressed: 0 bytes in 0 blocks ==2355120== Rerun with --leak-check=full to see details of leaked memory ==2355120== ==2355120== For lists of detected and suppressed errors, rerun with: -s ==2355120== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
We find that in this version, there exists memory leak issues.
Find the bug(Bonus)
Why does this problem happen? Use powerful git diff and gdb(you can add flags you want in the project Makefile) to find the bug!
You can follow the instruction of README to install the latest version of readline to eliminate the bug, or do not fix it and avoid such a usage. I prefer the latter to keep the stablity of package dependencies.
Acknowledgement
Thanks Geng Tiancheng(top-mind) for discovering the bug and telling it to me at the first time, as well as delegating me to complete such a blog.