Please see my comments to the question. Again, you are approaching it in a wrong way.
When I replied that using big text files is a bad thing and asked about your goals, you did not really explain them, but you mentioned that "some sort of software" which probably doesn't give you a choice. But then, why asking about "decreasing a size"? Who is going to decrease it? Isn't that logical.
And still, we don't know essential information, structure and semantic of the file. Okay, this is one of possible approaches:
You can index the file, to introduce the ability to read it by smaller chunks. Let's assume the file has some shallow structure; in particular, it would mean it can be decomposed on some smaller logical chunks we shall call "records". A record can be a line, but it could be a group of lines, like the group you've shown in your example. Then only problem then is that each group has different size; first of all, all lines have different size, so you don't know the location of each record before you read the whole file.
So, on first run, you can read the whole file line by line and create another, smaller file, the index file. In index file, you can write the location of each record as file position. It would be better to make the index file binary, to navigate faster in that file. You can have more then one index file, sorted by different criteria (one is sorted by record number in the order defined in the original file another one sorted by some kind of keyword, for example). Then, you can hold the index file in memory, and, if even the index files are big, store the only index of the index file, and read the index files on request.
Now, on request/query, you get the information on some record from one or another index file (take from memory or read from index file). From index information, get the position in the main original big file and seek this position in the file stream (open it once and keep open during the whole lifetime of the application). Then read your record from the original file.
One slightly different alternative: do everything as described above, but, on first run, completely rewrite the original text file in something more convenient for navigation, which could be much shorter binary file. In that binary file, don't store numbers as strings; it will save you a lot of space and, more importantly, greatly improve your performance.
—SA
Submit an article or tip