Fix data loss when reading large transaction bodies (#495)
The `read_from_stream` function in pgcopydb was encountering a data loss issue
when reading from a source system with transaction bodies containing more than
1024 inserts. The problem arose from the statically allocated array of size 1024
used to split lines within the function. When the buffer held more than 1024
lines, it caused subsequent lines to be ignored, resulting in data loss.
To resolve this issue, this commit introduces the following changes:
- We now count the number of lines in the buffer before splitting them.
- Dynamically allocate an array to hold the split lines, based on the line count.
Signed-off-by:
Arunprasad Rajkumar <ar.arunprasad@gmail.com>
parent
b1a0bcfe
Please register or sign in to comment