Split a file into several files based on datestamp matching

Get help with installation and running here.

Moderators: DataMystic Support, Moderators

Posts: 1
Joined: Thu Sep 30, 2004 2:45 am

Split a file into several files based on datestamp matching

Postby yogi » Thu Sep 30, 2004 2:55 am


I have a huge file containing history of all trades for a given stock for a year. Say, the stock is MSFT (microsoft). Each line/record begins with a datestamp of the format: MM/DD/YYYY. Each day has variable number of trades and the number of trades per day is in thousands.

Assuming there are 200 days, I would like to split the huge file into 200 (or so) files, each file containing the history of all trades for one day.

So far, I have been able to specify a date, say 29 Dec. 2000 using a "Retain lines matching perl pattern" filter and have been able to extract a day's data and save it. Then I go and rename the file to MSFT29Dec.TXT.

Is it possible to do the following two things in TextPipe:
1. Specify a generic date filter and have the large input file split into multiple files based on each trading day of the year?
2. Specify date as part of the output file's name so as to obtain 200 or so files after the operation is complete?

Would appreciate any pointers.

User avatar
DataMystic Support
Site Admin
Posts: 2138
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia

Postby DataMystic Support » Wed Oct 06, 2004 11:48 am


Use an EasyPattern match to identify date changes, and insert a special character which a subsequent split filter will split the file on,

Next you use a Restrict to Filename filter with a VBScript filter inside it to set the filename.

Simon Carter, http://DataMystic.com/forums/index.php
http://PredictBGL.com - Insulin dose calculator for Type 1 diabetes
http://DownloadPipe.com - 250,000 free software downloads
http://DetachPipe.com - send huge email attachments

Return to “TextPipe Tips and Tricks, Questions and Support”

Who is online

Users browsing this forum: No registered users and 5 guests