Reading Text Files Line by Line in C#

Solarian Programmer

My programming ramblings

C Programming - read a file line by line with fgets and getline, implement a portable getline version

Posted on April 3, 2019 past Paul

In this article, I volition show yous how to read a text file line by line in C using the standard C office fgets and the POSIX getline function. At the stop of the article, I volition write a portable implementation of the getline function that can be used with whatever standard C compiler.

Reading a file line past line is a trivial trouble in many programming languages, but not in C. The standard way of reading a line of text in C is to utilise the fgets function, which is fine if you know in advance how long a line of text could be.

You can find all the lawmaking examples and the input file at the GitHub repo for this article.

Let's kickoff with a simple case of using fgets to read chunks from a text file. :

                                                                        1                                    #include                  <stdio.h>                                                        2                                    #include                  <stdlib.h>                                                        3                                                        4                                    int                  main                  (                  void                  )                  {                                      5                                    FILE                  *                  fp                  =                  fopen                  (                  "lorem.txt"                  ,                  "r"                  );                                      6                                    if                  (                  fp                  ==                  NULL                  )                  {                                      7                                    perror                  (                  "Unable to open file!"                  );                                      eight                                    go out                  (                  1                  );                                      9                                    }                  10                                    xi                                    char                  clamper                  [                  128                  ];                  12                                    13                                    while                  (                  fgets                  (                  chunk                  ,                  sizeof                  (                  chunk                  ),                  fp                  )                  !=                  Goose egg                  )                  {                  14                                    fputs                  (                  chunk                  ,                  stdout                  );                  15                                    fputs                  (                  "|*                  \n                  "                  ,                  stdout                  );                  // marker string used to prove where the content of the chunk array has ended                  16                                    }                  17                                    xviii                                    fclose                  (                  fp                  );                  19                                    }                              

For testing the code I've used a simple dummy file, lorem.txt. This is a piece from the output of the to a higher place program on my motorcar:

                                                                        1                                    ~ $ clang -std=c17 -Wall -Wextra -pedantic t0.c -o t0                                      2                                    ~ $ ./t0                                      3                                    Lorem ipsum dolor sit down amet, consectetur adipiscing elit.                                      4                                    |*                                      v                                    Fusce dignissim facilisis ligula consectetur hendrerit. Vestibulum porttitor aliquam luctus. Nam pharetra lorem vel ornare cond|*                                      6                                    imentum.                                      7                                    |*                                      8                                    Praesent et nunc at libero vulputate convallis. Cras egestas nunc vitae eros vehicula hendrerit. Pellentesque in est et sapien |*                                      nine                                    dignissim molestie.                  10                                    |*                              

The lawmaking prints the content of the clamper array, as filled after every call to fgets, and a marker string.

If you lot watch carefully, by scrolling the above text snippet to the right, yous can run into that the output was truncated to 127 characters per line of text. This was expected because our code can shop an entire line from the original text file only if the line tin can fit inside our chunk assortment.

What if you need to take the entire line of text bachelor for further processing and not a piece of line ? A possible solution is to copy or concatenate chunks of text in a separate line buffer until nosotros discover the end of line character.

Let'south start past creating a line buffer that will store the chunks of text, initially this will have the same length as the clamper array:

                                                                        one                                    #include                  <stdio.h>                                                        ii                                    #include                  <stdlib.h>                                                        3                                    #include                  <string.h>                                                        4                                                        5                                    int                  main                  (                  void                  )                  {                                      half-dozen                                    FILE                  *                  fp                  =                  fopen                  (                  "lorem.txt"                  ,                  "r"                  );                                      7                                    // ...                                      8                                                        9                                    char                  chunk                  [                  128                  ];                  x                                    11                                    // Store the chunks of text into a line buffer                  12                                    size_t                  len                  =                  sizeof                  (                  chunk                  );                  13                                    char                  *                  line                  =                  malloc                  (                  len                  );                  14                                    if                  (                  line                  ==                  NULL                  )                  {                  15                                    perror                  (                  "Unable to allocate memory for the line buffer."                  );                  16                                    exit                  (                  i                  );                  17                                    }                  xviii                                    19                                    // "Empty" the string                  twenty                                    line                  [                  0                  ]                  =                  '\0'                  ;                  21                                    22                                    // ...                  23                                    24                                    }                              

Next, we are going to append the content of the clamper array to the end of the line string, until nosotros find the cease of line character. If necessary, we'll resize the line buffer:

                                                                        one                                    #include                  <stdio.h>                                                        2                                    #include                  <stdlib.h>                                                        iii                                    #include                  <cord.h>                                                        four                                                        5                                    int                  main                  (                  void                  )                  {                                      6                                    // ...                                      7                                                        8                                    // "Empty" the string                                      ix                                    line                  [                  0                  ]                  =                  '\0'                  ;                  10                                    11                                    while                  (                  fgets                  (                  chunk                  ,                  sizeof                  (                  chunk                  ),                  fp                  )                  !=                  NULL                  )                  {                  12                                    // Resize the line buffer if necessary                  13                                    size_t                  len_used                  =                  strlen                  (                  line                  );                  14                                    size_t                  chunk_used                  =                  strlen                  (                  chunk                  );                  15                                    xvi                                    if                  (                  len                  -                  len_used                  <                  chunk_used                  )                  {                  17                                    len                  *=                  2                  ;                  eighteen                                    if                  ((                  line                  =                  realloc                  (                  line                  ,                  len                  ))                  ==                  NULL                  )                  {                  19                                    perror                  (                  "Unable to reallocate retentivity for the line buffer."                  );                  20                                    gratis                  (                  line                  );                  21                                    exit                  (                  one                  );                  22                                    }                  23                                    }                  24                                    25                                    // Copy the chunk to the finish of the line buffer                  26                                    strncpy                  (                  line                  +                  len_used                  ,                  chunk                  ,                  len                  -                  len_used                  );                  27                                    len_used                  +=                  chunk_used                  ;                  28                                    29                                    // Check if line contains '\northward', if yes process the line of text                  thirty                                    if                  (                  line                  [                  len_used                  -                  1                  ]                  ==                  '\n'                  )                  {                  31                                    fputs                  (                  line                  ,                  stdout                  );                  32                                    fputs                  (                  "|*                  \northward                  "                  ,                  stdout                  );                  33                                    // "Empty" the line buffer                  34                                    line                  [                  0                  ]                  =                  '\0'                  ;                  35                                    }                  36                                    }                  37                                    38                                    fclose                  (                  fp                  );                  39                                    free                  (                  line                  );                  forty                                    41                                    printf                  (                  "                  \n\north                  Max line size: %zd                  \n                  "                  ,                  len                  );                  42                                    }                              

Please annotation, that in the above code, every time the line buffer needs to be resized its capacity is doubled.

This is the effect of running the above code on my machine. For brevity, I kept merely the first lines of output:

                                                                        one                                    ~ $ clang -std=c17 -Wall -Wextra -pedantic t1.c -o t1                                      2                                    ~ $ ./t1                                      3                                    Lorem ipsum dolor sit down amet, consectetur adipiscing elit.                                      iv                                    |*                                      v                                    Fusce dignissim facilisis ligula consectetur hendrerit. Vestibulum porttitor aliquam luctus. Nam pharetra lorem vel ornare condimentum.                                      six                                    |*                                      seven                                    Praesent et nunc at libero vulputate convallis. Cras egestas nunc vitae eros vehicula hendrerit. Pellentesque in est et sapien dignissim molestie.                                      8                                    |*                                      9                                    Aliquam erat volutpat. Mauris dignissim augue ac purus placerat scelerisque. Donec eleifend ut nibh eu elementum.                  10                                    |*                              

Yous can meet that, this time, we can impress total lines of text and not fixed length chunks similar in the initial approach.

Permit's modify the above lawmaking in lodge to print the line length instead of the bodily text:

                                                                        1                                    // ...                                      2                                                        3                                    int                  main                  (                  void                  )                  {                                      4                                    // ...                                      5                                                        half dozen                                    while                  (                  fgets                  (                  chunk                  ,                  sizeof                  (                  chunk                  ),                  fp                  )                  !=                  Null                  )                  {                                      7                                                        eight                                    // ...                                      9                                    10                                    // Check if line contains '\n', if yes process the line of text                  11                                    if                  (                  line                  [                  len_used                  -                  i                  ]                  ==                  '\north'                  )                  {                  12                                    printf                  (                  "line length: %zd                  \n                  "                  ,                  len_used                  );                  13                                    // "Empty" the line buffer                  14                                    line                  [                  0                  ]                  =                  '\0'                  ;                  15                                    }                  16                                    }                  17                                    18                                    fclose                  (                  fp                  );                  19                                    free                  (                  line                  );                  20                                    21                                    printf                  (                  "                  \north\n                  Max line size: %zd                  \n                  "                  ,                  len                  );                  22                                    }                              

This is the result of running the modified lawmaking on my machine:

                                                                        1                                    ~ $ clang -std=c17 -Wall -Wextra -pedantic t1.c -o t1                                      2                                    ~ $ ./t1                                      three                                    line length: 57                                      iv                                    line length: 136                                      5                                    line length: 147                                      half dozen                                    line length: 114                                      7                                    line length: 112                                      eight                                    line length: 95                                      nine                                    line length: 62                  10                                    line length: 1                  11                                    line length: 428                  12                                    line length: i                  xiii                                    line length: 460                  14                                    line length: i                  15                                    line length: 834                  xvi                                    line length: ane                  17                                    line length: 821                  18                                    nineteen                                    20                                    Max line size: 1024                              

In the adjacent example, I will show you how to use the getline function available on POSIX systems like Linux, Unix and macOS. Microsoft Visual Studio doesn't accept an equivalent function, and so y'all won't be able to easily test this instance on a Windows organisation. Still, you should be able to test it if you lot are using Cygwin or Windows Subsystem for Linux.

                                                                        1                                    #include                  <stdio.h>                                                        2                                    #include                  <stdlib.h>                                                        3                                    #include                  <string.h>                                                        4                                                        v                                    int                  main                  (                  void                  )                  {                                      six                                    FILE                  *                  fp                  =                  fopen                  (                  "lorem.txt"                  ,                  "r"                  );                                      seven                                    if                  (                  fp                  ==                  Nix                  )                  {                                      8                                    perror                  (                  "Unable to open file!"                  );                                      9                                    leave                  (                  1                  );                  10                                    }                  11                                    12                                    // Read lines using POSIX function getline                  xiii                                    // This code won't work on Windows                  14                                    char                  *                  line                  =                  NULL                  ;                  15                                    size_t                  len                  =                  0                  ;                  16                                    17                                    while                  (                  getline                  (                  &                  line                  ,                  &                  len                  ,                  fp                  )                  !=                  -                  1                  )                  {                  eighteen                                    printf                  (                  "line length: %zd                  \north                  "                  ,                  strlen                  (                  line                  ));                  nineteen                                    }                  20                                    21                                    printf                  (                  "                  \northward\n                  Max line size: %zd                  \due north                  "                  ,                  len                  );                  22                                    23                                    fclose                  (                  fp                  );                  24                                    gratuitous                  (                  line                  );                  // getline will resize the input buffer as necessary                  25                                    // the user needs to free the retentiveness when not needed!                  26                                    }                              

Delight note, how simple is to use POSIX's getline versus manually buffering chunks of line like in my previous case. It is unfortunate that the standard C library doesn't include an equivalent function.

When yous use getline, don't forget to free the line buffer when y'all don't need information technology anymore. Too, calling getline more than than once will overwrite the line buffer, brand a copy of the line content if y'all need to keep it for further processing.

This is the outcome of running the in a higher place getline example on a Linux machine:

                                                                        1                                    ~ $ clang -std=gnu17 -Wall -Wextra -pedantic t2.c -o t2                                      ii                                    ~ $ ./t2                                      three                                    line length: 57                                      four                                    line length: 136                                      v                                    line length: 147                                      6                                    line length: 114                                      vii                                    line length: 112                                      viii                                    line length: 95                                      9                                    line length: 62                  ten                                    line length: ane                  11                                    line length: 428                  12                                    line length: 1                  13                                    line length: 460                  14                                    line length: 1                  15                                    line length: 834                  16                                    line length: 1                  17                                    line length: 821                  18                                    19                                    20                                    Max line size: 960                              

It is interesting to note, that for this detail case the getline function on Linux resizes the line buffer to a max of 960 bytes. If you lot run the same code on macOS the line buffer is resized to 1024 bytes. This is due to the different ways in which getline is implemented on different Unix similar systems.

Every bit mentioned before, getline is not nowadays in the C standard library. Information technology could be an interesting do to implement a portable version of this function. The thought here is not to implement the most performant version of getline, but rather to implement a unproblematic replacement for not POSIX systems.

We are going to take the higher up instance and replace the POSIX's getline version with our own implementation, say my_getline. Apparently, if you are on a POSIX system, you lot should apply the version provided by the operating system, which was tested by countless users and tuned for optimal operation.

The POSIX getline part has this signature:

                                                    1                                    ssize_t                  getline                  (                  char                  **                  restrict                  lineptr                  ,                  size_t                  *                  restrict                  northward                  ,                  FILE                  *                  restrict                  stream                  );                              

Since ssize_t is also a POSIX divers blazon, commonly a 64 bits signed integer, this is how we are going to declare our version:

                                                    1                                    int64_t                  my_getline                  (                  char                  **                  restrict                  line                  ,                  size_t                  *                  restrict                  len                  ,                  FILE                  *                  restrict                  fp                  );                              

In principle we are going to implement the role using the same approach as in 1 of the above examples, where I've defined a line buffer and kept copying chunks of text in the buffer until nosotros found the finish of line graphic symbol:

                                                                        one                                    // This volition only have effect on Windows with MSVC                                      ii                                    #ifdef _MSC_VER                                      three                                    #define _CRT_SECURE_NO_WARNINGS i                                      4                                    #define restrict __restrict                                      5                                    

Share this post