This system is being developed and used by NASA at the Kennedy Space Center.
The master file data was not accessible via the tools provided with the system and there was no way to build a long term history. Without this information building a long term archival and retrieval system would be impossible. There was a need to sort through thousands of files and find the correct information.
TextPipe allowed us to build a filter that would find the information within these binary files and send it to another file for processing without writing our own code.
The SCADA software creates a master history file (.hst) and history data files for each trended value. The number of files is set by the builder based on duration of online history desired. We desire 90 days using 90 files, one file for 24 hours. Our system will have about 10,000 trend values in the near term. This will equal 91 x 10,000 files. The SCADA system software calls TexPipe via a command line and executes the filter. TextPipe then quickly reads all .hst files to discern the order of the history files. This order is then written to a separate file. The SCADA software then reads this file and selects the correct file to send to a long term data archive.