A memory leak issue which might be confusing for students struggling
with ICS Project (aka, PA) of Nanjing
University.
Notification
The issue mentioned in this article has been discovered by the
developer community for a while(And probabily been fixed so far). But
for freshmen in this field, such as students taking NJU ICS course, this
is not well-known and may cause serious confusion. Geng
Tiancheng(top-mind) discovered this during his implementing
of PA(GitHub
issue page). And he delegated me, Wenrui
Huang(rijuyuezhu), to write a blog recording the issue, in
order to help students who meet the same situation.
Bug Summary
The environment
All the tests are based on the environment listed below.
$ apt show libreadline8 Package: libreadline8 Version: 8.1.2-1 Priority: important Section: libs Source: readline Origin: Ubuntu Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com> Original-Maintainer: Matthias Klose <doko@debian.org> Bugs: https://bugs.launchpad.net/ubuntu/+filebug Installed-Size: 472 kB Depends: readline-common, libc6 (>= 2.33), libtinfo6 (>= 6) Task: minimal, server-minimal Download-Size: 153 kB APT-Manual-Installed: no APT-Sources: https://mirror.nju.edu.cn/ubuntu jammy/main amd64 Packages Description: GNU readline and history libraries, run-time libraries The GNU readline library aids in the consistency of user interface across discrete programs that need to provide a command line interface. . The GNU history library provides a consistent user interface for recalling lines of previously typed input.
Overview
When following the instruction of PA gitbook: PA1 ~ 监视点 ~ sanitizer - 一种底层的assert,
(find
the page in ICS2023), students turn on the functionality of address
sanitizer. This tools help to provide basic examinations of array access
out of bounds as well as memory leaks.
However, though this tool helps a lot dealing with memory leak issues
caused by the students(for example, non-matching malloc and
free), some of the students will meet strange memory leak
fault that indicating there is an issue in the lib
readline, like the message below:
Direct leak of 32 byte(s) in 1 object(s) allocated from: #0 0x7f9b452b4887 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145 #1 0x7f9b45c5dbac in xmalloc (/lib/x86_64-linux-gnu/libreadline.so.8+0x39bac)
Direct leak of 32 byte(s) in 1 object(s) allocated from: #0 0x7f73104b4887 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145 #1 0x7f7310f5fbac in xmalloc (/lib/x86_64-linux-gnu/libreadline.so.8+0x39bac)
Indirect leak of 402 byte(s) in 21 object(s) allocated from: #0 0x7f73104b4887 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145 #1 0x7f7310f5fbac in xmalloc (/lib/x86_64-linux-gnu/libreadline.so.8+0x39bac)
We learn that this issue is related to the history functionality of
readline lib. However, because the shared library is not
compiled with flag -g, and gdb also cannot find the
position of memory leak, the debug process may be hard, even though we
know it is the library readline to blame.
Research the source
repo of lib readline
It is great that readline is open-source, and we can easily obtain
the source code of the library. Links are as follows:
$ git log --oneline --graph * 7274faa (HEAD -> master, origin/master, origin/HEAD) Readline-8.2 patch 1: fix crash when readline is started with an invalid locale specification * f7a382f (tag: readline-8.2) readline-8.2 distribution sources and documentation * 5263c0d Readline-8.1 patch 2: fix redisplay of some characters > 128 in certain single-byte encodings * 9ba3434 Readline-8.1 patch 1: fix version comparisons in startup files * cf3c762 (tag: readline-8.1) Readline-8.1 distribution sources and documentation * c5ad6be problems restoring the history file are not signaled correctly to the calling application ...
Some developer reports that the bug is fixed in the version
readline-8.2, and let us verify that. Switch to the
readline-8.2 tag and compile the lib (according to
README)
1 2 3
$ git checkout readline-8.2 $ ./configure $ make
The readline lib provides some examples, and we can use that for our
bug reproduction.
1 2
$ cd examples $ make
We can use, for example, rltest for test. But to detect
memory leak, we need some tools. Valgrind is a very powerful tool for
such a need. Run rltest with valgrind:
1 2 3 4 5 6 7
$ valgrind ./rltest ==2341346== Memcheck, a memory error detector ==2341346== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al. ==2341346== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info ==2341346== Command: ./rltest ==2341346== readline$
Remember what to do to reproduce the bug? First input random string
and enter <CR>, then input random string and enter up arrow, and
finally enter <CR> and quit the program(We can use Ctrl-C
here):
$ valgrind ./rltest ==2341346== Memcheck, a memory error detector ==2341346== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al. ==2341346== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info ==2341346== Command: ./rltest ==2341346== readline$ abc abc readline$ abc abc # Note: this string is initially not abc, but turns to abc after entering up arrow ^C ==2341346== ==2341346== <Some other info> ==2341346== ==2341346== LEAK SUMMARY: ==2341346== definitely lost: 0 bytes in 0 blocks ==2341346== indirectly lost: 0 bytes in 0 blocks ==2341346== possibly lost: 0 bytes in 0 blocks ==2341346== still reachable: 222,888 bytes in 271 blocks ==2341346== suppressed: 0 bytes in 0 blocks ==2341346== Rerun with --leak-check=full to see details of leaked memory ==2341346== ==2341346== For lists of detected and suppressed errors, rerun with: -s ==2341346== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
We find that the memory lost is zero, so the bug is fixed in
readline-8.2.
And we checkout the former version:
1 2 3
$ git checkout HEAD~ Previous HEAD position was f7a382f readline-8.2 distribution sources and documentation HEAD is now at 5263c0d Readline-8.1 patch 2: fix redisplay of some characters > 128 in certain single-byte encodings
And do the same test as above, we get info like:
1 2 3 4 5 6 7 8 9 10
==2355120== LEAK SUMMARY: ==2355120== definitely lost: 32 bytes in 1 blocks ==2355120== indirectly lost: 66 bytes in 3 blocks ==2355120== possibly lost: 0 bytes in 0 blocks ==2355120== still reachable: 198,179 bytes in 249 blocks ==2355120== suppressed: 0 bytes in 0 blocks ==2355120== Rerun with --leak-check=full to see details of leaked memory ==2355120== ==2355120== For lists of detected and suppressed errors, rerun with: -s ==2355120== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
We find that in this version, there exists memory leak issues.
Find the bug(Bonus)
Why does this problem happen? Use powerful git diff and
gdb(you can add flags you want in the project Makefile) to
find the bug!
You can follow the instruction of README to install the latest
version of readline to eliminate the bug, or do not fix it
and avoid such a usage. I prefer the latter to keep the stablity of
package dependencies.
Acknowledgement
Thanks Geng
Tiancheng(top-mind) for discovering the bug and telling it
to me at the first time, as well as delegating me to complete such a
blog.