hn-classics/_stories/2010/8410545.md

372 lines
17 KiB
Markdown
Raw Permalink Normal View History

---
created_at: '2014-10-04T20:26:15.000Z'
title: Where the Printf Rubber Meets the Road (2010)
url: http://blog.hostilefork.com/where-printf-rubber-meets-road/
author: striking
points: 80
story_text: ''
comment_text:
num_comments: 44
story_id:
story_title:
story_url:
parent_id:
created_at_i: 1412454375
_tags:
- story
- author_striking
- story_8410545
objectID: '8410545'
2018-06-08 12:05:27 +00:00
year: 2010
---
2018-02-23 18:19:40 +00:00
[Source](http://blog.hostilefork.com/where-printf-rubber-meets-road/ "Permalink to c : Where the printf() Rubber Meets the Road")
# c : Where the printf() Rubber Meets the Road
![Feed Icon][1] [RSS 1.0 XML Feed][2] available
# Where the printf() Rubber Meets the Road
* [Home][3]
* [c][4]
* Where the printf() Rubber Meets the Road
**Date:** 14-Mar-2010/19:52
**Tags:** [c][4], [stackexchange][5]
**Characters:** (none)
After ignoring StackOverflow for a while, I decided to check up on it a bit lately. Someone asked [a question][6] that's one of those kind of fundamental curiosity issues that I enjoy explaining. He said:
> I always thought that functions like printf() are in the last step defined using inline assembly. That deep into stdio.h is buried some asm code that actually tells CPU what to do. Something like in dos, first mov beginning of the string to some memory location or register and than call some int. But since x64 version of Visual Studio doesn't support inline assembler at all, it made me think that there are really no assembler-defined functions in C/C++. So, please, how is for example printf() defined in C/C++ without using assembler code? What actually executes the right software interrupt?
Obviously the answer is going to depend on the implementation. Yet I thought that with the open-sourced GNU C Library, it would be pretty straightforward to show how most of it is in C but it bottoms out at `syscall`. But it really was quite a maze to connect all the dots without doing any hand-waving! So I found that my explanation just kept growing until it was so long that a blog entry was a more fitting format.
So read on, fearless explorers, as we dig into the complicated answer to a seemingly simple question...
### First Steps
We'll of course start with the prototype for `printf`, which is defined in the file [libc/libio/stdio.h][7]
extern int printf (__const char *__restrict __format, ...);
You won't find the source code for a function called `printf`, however. Instead, in the file [/libc/stdio-common/printf.c][8] you'll find a little bit of code associated with a function called `__printf`:
int __printf (const char *format, ...)
{
va_list arg;
int done;
va_start (arg, format);
done = vfprintf (stdout, format, arg);
va_end (arg);
return done;
}
A macro in the same file sets up an association so that this function is defined as an alias for the non-underscored `printf`:
ldbl_strong_alias (__printf, printf);
It makes sense that `printf` would be a thin layer that calls `vfprintf` with `stdout`. Indeed, the meat of the formatting work is done in `vfprintf`, which you'll find in [libc/stdio-common/vfprintf.c][9]. It's quite a lengthy function, but you can see that it's still all in C!
### Deeper Down the Rabbit Hole...
`vfprintf` mysteriously calls `outchar` and `outstring`, which are weird macros defined in the same file:
#define outchar(Ch)
do
{
register const INT_T outc = (Ch);
if (PUTC (outc, s) == EOF || done == INT_MAX)
{
done = -1;
goto all_done;
}
++done;
}
while (0)
Sidestepping the question of why it's so weird, we see that it's dependent on the enigmatic `PUTC`, also in the same file:
#define PUTC(C, F) IO_putwc_unlocked (C, F)
When you get to the definition of `IO_putwc_unlocked` in [libc/libio/libio.h][10], you might start thinking that you no longer care how `printf` works:
#define _IO_putwc_unlocked(_wch, _fp)
(_IO_BE ((_fp)->_wide_data->_IO_write_ptr
>= (_fp)->_wide_data->_IO_write_end, 0)
? __woverflow (_fp, _wch)
: (_IO_wint_t) (*(_fp)->_wide_data->_IO_write_ptr++ = (_wch)))
But despite being a little hard to read, it's just doing buffered output. If there's enough room in the file pointer's buffer, then it will just stick the character into it...but if not, it calls `__woverflow`. Since the only option when you've run out of buffer is to flush to the screen (or whatever device your file pointer represents), we can hope to find the magic incantation there.
### Vtables in C?
If you guessed that we're going to hop through another frustrating level of indirection, you'd be right. Look in [libc/libio/wgenops.c][11] and you'll find the definition of `__woverflow`:
wint_t
__woverflow (f, wch)
_IO_FILE *f;
wint_t wch;
{
if (f->_mode == 0)
_IO_fwide (f, 1);
return _IO_OVERFLOW (f, wch);
}
Basically, file pointers are implemented in the GNU standard library as objects. They have data members but also function members which you can call with variations of the `JUMP` macro. In the file [libc/libio/libioP.h][12] you'll find a little documentation of this technique:
/* THE JUMPTABLE FUNCTIONS.
* The _IO_FILE type is used to implement the FILE type in GNU libc,
* as well as the streambuf class in GNU iostreams for C++.
* These are all the same, just used differently.
* An _IO_FILE (or FILE) object is allows followed by a pointer to
* a jump table (of pointers to functions). The pointer is accessed
* with the _IO_JUMPS macro. The jump table has a eccentric format,
* so as to be compatible with the layout of a C++ virtual function table.
* (as implemented by g++). When a pointer to a streambuf object is
* coerced to an (_IO_FILE*), then _IO_JUMPS on the result just
* happens to point to the virtual function table of the streambuf.
* Thus the _IO_JUMPS function table used for C stdio/libio does
* double duty as the virtual function table for C++ streambuf.
*
* The entries in the _IO_JUMPS function table (and hence also the
* virtual functions of a streambuf) are described below.
* The first parameter of each function entry is the _IO_FILE/streambuf
* object being acted on (i.e. the 'this' parameter).
*/
So when we find `IO_OVERFLOW` in [libc/libio/genops.c][13], we find it's a macro which calls a "1-parameter" `__overflow` method on the file pointer:
#define IO_OVERFLOW(FP, CH) JUMP1 (__overflow, FP, CH)
The jump tables for the various file pointer types are in [libc/libio/fileops.c][14]
const struct _IO_jump_t _IO_file_jumps =
{
JUMP_INIT_DUMMY,
JUMP_INIT(finish, INTUSE(_IO_file_finish)),
JUMP_INIT(overflow, INTUSE(_IO_file_overflow)),
JUMP_INIT(underflow, INTUSE(_IO_file_underflow)),
JUMP_INIT(uflow, INTUSE(_IO_default_uflow)),
JUMP_INIT(pbackfail, INTUSE(_IO_default_pbackfail)),
JUMP_INIT(xsputn, INTUSE(_IO_file_xsputn)),
JUMP_INIT(xsgetn, INTUSE(_IO_file_xsgetn)),
JUMP_INIT(seekoff, _IO_new_file_seekoff),
JUMP_INIT(seekpos, _IO_default_seekpos),
JUMP_INIT(setbuf, _IO_new_file_setbuf),
JUMP_INIT(sync, _IO_new_file_sync),
JUMP_INIT(doallocate, INTUSE(_IO_file_doallocate)),
JUMP_INIT(read, INTUSE(_IO_file_read)),
JUMP_INIT(write, _IO_new_file_write),
JUMP_INIT(seek, INTUSE(_IO_file_seek)),
JUMP_INIT(close, INTUSE(_IO_file_close)),
JUMP_INIT(stat, INTUSE(_IO_file_stat)),
JUMP_INIT(showmanyc, _IO_default_showmanyc),
JUMP_INIT(imbue, _IO_default_imbue)
};
libc_hidden_data_def (_IO_file_jumps)
There's also a #define which equates `_IO_new_file_overflow` with `_IO_file_overflow`, and the former is defined in the same source file.
Note `INTUSE` is just a macro which marks functions that are for internal use, it doesn't mean anything like "this function uses an interrupt")
### Are we there yet?!
The source code for `_IO_new_file_overflow` does a bunch more buffer manipulation, but it does call `_IO_do_flush`:
#define _IO_do_flush(_f)
INTUSE(_IO_do_write)(_f, (_f)->_IO_write_base,
(_f)->_IO_write_ptr-(_f)->_IO_write_base)
We're now at a point where `_IO_do_write` is probably where the rubber actually meets the road: an _unbuffered_, _actual_, _direct_ write to an I/O device. At least we can hope! It is mapped by a macro to `_IO_new_do_write` and we have this:
static
_IO_size_t
new_do_write (fp, data, to_do)
_IO_FILE *fp;
const char *data;
_IO_size_t to_do;
{
_IO_size_t count;
if (fp->_flags & _IO_IS_APPENDING)
/* On a system without a proper O_APPEND implementation,
you would need to sys_seek(0, SEEK_END) here, but is
is not needed nor desirable for Unix- or Posix-like systems.
Instead, just indicate that offset (before and after) is
unpredictable. */
fp->_offset = _IO_pos_BAD;
else if (fp->_IO_read_end != fp->_IO_write_base)
{
_IO_off64_t new_pos
= _IO_SYSSEEK (fp, fp->_IO_write_base - fp->_IO_read_end, 1);
if (new_pos == _IO_pos_BAD)
return 0;
fp->_offset = new_pos;
}
count = _IO_SYSWRITE (fp, data, to_do);
if (fp->_cur_column && count)
fp->_cur_column = INTUSE(_IO_adjust_column) (fp->_cur_column - 1, data,
- count) + 1;
_IO_setg (fp, fp->_IO_buf_base, fp->_IO_buf_base, fp->_IO_buf_base);
fp->_IO_write_base = fp->_IO_write_ptr = fp->_IO_buf_base;
fp->_IO_write_end = (fp->_mode <= 0
&& (fp->_flags & (_IO_LINE_BUF+_IO_UNBUFFERED))
? fp->_IO_buf_base : fp->_IO_buf_end);
return count;
}
Sadly we're stuck again... `_IO_SYSWRITE` is doing the work:
/* The 'syswrite' hook is used to write data from an existing buffer
to an external file. It generalizes the Unix write(2) function.
It matches the streambuf::sys_write virtual function, which is
specific to this implementation. */
typedef _IO_ssize_t (*_IO_write_t) (_IO_FILE *, const void *, _IO_ssize_t);
#define _IO_SYSWRITE(FP, DATA, LEN) JUMP2 (__write, FP, DATA, LEN)
#define _IO_WSYSWRITE(FP, DATA, LEN) WJUMP2 (__write, FP, DATA, LEN)
So inside of the `do_write` we call the `write` method on the file pointer. We know from our jump table above that is mapped to `_IO_new_file_write`, so what's that do?
_IO_ssize_t
_IO_new_file_write (f, data, n)
_IO_FILE *f;
const void *data;
_IO_ssize_t n;
{
_IO_ssize_t to_do = n;
while (to_do > 0)
{
_IO_ssize_t count = (__builtin_expect (f->_flags2
& _IO_FLAGS2_NOTCANCEL, 0)
? write_not_cancel (f->_fileno, data, to_do)
: write (f->_fileno, data, to_do));
if (count < 0)
{
f->_flags |= _IO_ERR_SEEN;
break;
}
to_do -= count;
data = (void *) ((char *) data + count);
}
n -= to_do;
if (f->_offset >= 0)
f->_offset += n;
return n;
}
Now it just calls `write`! Well where is the implementation for that? You'll find `write` in [libc/posix/unistd.h][15]:
/* Write N bytes of BUF to FD. Return the number written, or -1.
This function is a cancellation point and therefore not marked with
__THROW. */
extern ssize_t write (int __fd, __const void *__buf, size_t __n) __wur;
Note `__wur` is a macro for `__attribute__ ((__warn_unused_result__))`
### Functions Generated From a Table
That's only a prototype for `write`. You won't find a write.c file for Linux in the GNU standard library. Instead, you'll find platform-specific methods of connecting to the OS write function in various ways, all in the [libc/sysdeps/][16] directory.
We'll keep following along with how Linux does it. There is a file called [sysdeps/unix/syscalls.list][17] which is used to generate the `write` function automatically. The relevant data from the table is:
* **File name:** write
* **Caller:** "-" _(i.e. Not Applicable)_
* **Syscall name:** write
* **Args:** Ci:ibn
* **Strong name:** __libc_write
* **Weak names: ** __write, write
Not all that mysterious, except for the **Ci:ibn**. The C means "cancellable". The colon separates the return type from the argument types, and if you want a deeper explanation of what they mean then you can see the comment in the shell script which generates the code, [libc/sysdeps/unix/make-syscalls.sh][18].
So now we're expecting to be able to link against a function called `__libc_write` which is generated by this shell script. But what's being generated? Some C code which implements write via a macro called `SYS_ify`, which you'll find in [sysdeps/unix/sysdep.h][19]
#define SYS_ify(syscall_name) __NR_##syscall_name
Ah, good old token-pasting :P. So basically, the implementation of this `__libc_write` becomes nothing more than a proxy invocation of the `syscall` function with a parameter named `__NR_write`, and the other arguments.
### Where The Sidewalk Ends...
I know this has been a fascinating journey, but now we're at the end of GNU libc. That number `__NR_write` is defined by Linux. For 32-bit X86 architectures it will get you to [linux/arch/x86/include/asm/unistd_32.h][20]:
#define __NR_write 4
This is the final step where the "rubber meets the road", but how it is done is system-dependent. That would make an article in itself, but we've found where the rubber meets the road. What you want to read up on to go further is this thing called ["syscall"][21].
[BIL 2010 PARTICIPANTS - WELCOME][22]
[When should one use const_cast, anyway?][23]
![Business Card from SXSW][24]
Copyright (c) 2007-2015 hostilefork.com
Project names and graphic designs are All Rights Reserved, unless otherwise noted. Software codebases are governed by licenses included in their distributions. Posts on [blog.hostilefork.com][25] are licensed under the Creative Commons [BY-NC-SA 4.0][26] license, and may be excerpted or adapted under the terms of that license for noncommercial purposes.
[1]: http://hostilefork.com/media/feed-icon-14x14.png
[2]: http://blog.hostilefork.com/feed/
[3]: http://blog.hostilefork.com/ "Home"
[4]: http://blog.hostilefork.com/tag/c/
[5]: http://blog.hostilefork.com/tag/stackexchange/
[6]: http://stackoverflow.com/questions/2442966/c-c-function-definitions-without-assembly/
[7]: http://sourceware.org/git/?p=glibc.git;a=blob;f=libio/stdio.h;h=85542b1cfdbd50941d288dde8e22e678b10b1333;hb=HEAD
[8]: http://sourceware.org/git/?p=glibc.git;a=blob;f=stdio-common/printf.c;h=4c8f3a2a0c38ab27a2eed4d2ff3b804980aa8f9f;hb=HEAD
[9]: http://sourceware.org/git/?p=glibc.git;a=blob;f=stdio-common/vprintf.c;h=5c9cac494a6db796a8263401f98964ae4b6a63a2;hb=HEAD
[10]: http://sourceware.org/git/?p=glibc.git;a=blob;f=libio/libio.h;h=3c9f2bd3e847429dcb14eb37d3b8369998bd19ee;hb=HEAD
[11]: http://sourceware.org/git/?p=glibc.git;a=blob;f=libio/wgenops.c;h=e2adedd6d4d07c9f66b870de644d6c3d84848063;hb=HEADp
[12]: http://sourceware.org/git/?p=glibc.git;a=blob;f=libio/libioP.h;h=854f049291c945b6ec2c552daefe901f23f95a4b;hb=HEAD
[13]: http://sourceware.org/git/?p=glibc.git;a=blob;f=libio/genops.c;h=481fbc52b0c0f7f0d4cabaa6676c48f92b84a093;hb=HEAD
[14]: http://sourceware.org/git/?p=glibc.git;a=blob;f=libio/fileops.c;h=4698953f7ae1d37b4e0911bf2e28cc72d7a1519e;hb=HEAD
[15]: http://sourceware.org/git/?p=glibc.git;a=blob;f=posix/unistd.h;h=f8b84e3cb35643264a4bfa42b8714c46976cf77c;hb=HEAD
[16]: http://sourceware.org/git/?p=glibc.git;a=tree;f=sysdeps;h=f1c8a1febcebabaf9f7b1529c7ca7c5a85ba98de;hb=HEAD
[17]: http://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/unix/syscalls.list;h=04ed63c4d75451b978fd9488e102fe0d058b0295;hb=HEAD
[18]: http://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/unix/make-syscalls.sh;h=a8b8a262a7cd20ee9fd4618ddc94120d283a813e;hb=HEAD
[19]: http://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/unix/sysdep.h;h=11e5317dfffdffd9ebc5ca2091d701b99be98b31;hb=HEAD
[20]: http://www.cs.fsu.edu/~baker/devices/lxr/http/source/linux/include/asm-x86/unistd_32.h?v=2.6.25#L12
[21]: http://en.wikipedia.org/wiki/System_call
[22]: http://blog.hostilefork.com/bil-2010-participants-welcome/ "Previous"
[23]: http://blog.hostilefork.com/when-should-one-use-const-cast/ "Next"
[24]: http://hostilefork.com/media/fork-card-back-320x560.png
[25]: http://blog.hostilefork.com
[26]: http://creativecommons.org/licenses/by-nc-sa/4.0/