Ah the good ol day's, this reminded me of some file transfer problems we used to get.
We had multiple systems that generated usage records, and stored them to flat file (think stuff that would end up on a bill). Because FTP the file was a thing, some other system would come in any copy the file, but every once in awhile there would be a partial file copied that would be missing records. Yep, it was the good ol process was still writing to the file when the collector decided to pick it up.
The first system I had control over, I made sure the vendor wrote to a temporary directory, and then hard linked to the transfer directory when the file operation was complete, knowing it avoided the race condition. I'm pretty sure I had one of the few platforms that handled this correctly, from what I remember we had corrupt files from almost all the platforms we bought.
Anyways, just because the system cost a million dollars doesn't mean it's any good.
We had multiple systems that generated usage records, and stored them to flat file (think stuff that would end up on a bill). Because FTP the file was a thing, some other system would come in any copy the file, but every once in awhile there would be a partial file copied that would be missing records. Yep, it was the good ol process was still writing to the file when the collector decided to pick it up.
The first system I had control over, I made sure the vendor wrote to a temporary directory, and then hard linked to the transfer directory when the file operation was complete, knowing it avoided the race condition. I'm pretty sure I had one of the few platforms that handled this correctly, from what I remember we had corrupt files from almost all the platforms we bought.
Anyways, just because the system cost a million dollars doesn't mean it's any good.