Beta Feedback - Split File by Pattern

Get help with installation and running here.

Moderators: DataMystic Support, Moderators

Bernard Toplak
Posts: 5
Joined: Fri Jun 16, 2006 12:13 am
Location: Varazdin, Croatia
Contact:

Beta Feedback - Split File by Pattern

Postby Bernard Toplak » Fri Jun 16, 2006 12:51 am

Hi to Simon and Datamystic Crew,

I am testing this new pattern function in Split File filter. It seems to work OK with text files, but on binaries it fails, making thousands of 0 byte files, and I can't find logical pattern why this happens.

I am testing it for extraction of PNG file fragments from one big binary file. When I simulate PNG start and end strings in plain textual file it splits perfectly, but in binary results as I said above.

I can send you my test files to see by yourself...
Last edited by Bernard Toplak on Fri Jun 16, 2006 4:52 pm, edited 1 time in total.

User avatar
DataMystic Support
Site Admin
Posts: 2136
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Postby DataMystic Support » Fri Jun 16, 2006 3:57 pm

Hi Bernard,

Ahem! We actually assumed that no-one would use it one binary files, hence we coded this in two parts
- a search replace for the pattern, which inserts a null character
- a normal split on nulls.

You can workaround this by removing the nulls first, which sounds like it would be useful anyway.
Regards,

Simon Carter, http://DataMystic.com/forums/index.php
http://PredictBGL.com - Insulin dose calculator for Type 1 diabetes
http://DownloadPipe.com - 250,000 free software downloads
http://DetachPipe.com - send huge email attachments

Bernard Toplak
Posts: 5
Joined: Fri Jun 16, 2006 12:13 am
Location: Varazdin, Croatia
Contact:

Postby Bernard Toplak » Fri Jun 16, 2006 4:47 pm

Simon Carter wrote:... we coded this in two parts
- a search replace for the pattern, which inserts a null character
- a normal split on nulls.

Well, with it's extreme pattern matching power TextPipe is ideal candidate for such jobs of splitting all kinds of files at pattern-principle.

I'm afraid that none of above methods is good for bitmap data as those files use full range of hex chars, so no split-on-char could be performed.

Simon Carter wrote:You can workaround this by removing the nulls first, which sounds like it would be useful anyway.

Hmm, you mean removing \x00 chars? I'm not sure that is a good solution when bitmap data is on the table. If the split on pattern would work with bins just like on text ones, extracting pattern match and ignoring any other unmatching data, without any additional character insertion it would work perfectly!

Bernard Toplak
Posts: 5
Joined: Fri Jun 16, 2006 12:13 am
Location: Varazdin, Croatia
Contact:

Postby Bernard Toplak » Tue Jun 27, 2006 5:32 am

Hi Simon & Crew!

I just wanted to ask if there will be any moves in this direction?

I have solved my task with one freeware tool that can identify more than 110 different file formats (and much more) in hex files. This can be very usefull for data forensic tasks like mine. If it is permited I can post info for other users that may need it because I see this is out of the TextPipe's main domain, right?

Best regards
Bernard

User avatar
DataMystic Support
Site Admin
Posts: 2136
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Postby DataMystic Support » Tue Jun 27, 2006 9:28 am

Hi Bernard,

Thanks for asking. Please email the details to me first.

You'll be please to know that we *will* be adding the full split on pattern function in the next day or so.
Regards,

Simon Carter, http://DataMystic.com/forums/index.php
http://PredictBGL.com - Insulin dose calculator for Type 1 diabetes
http://DownloadPipe.com - 250,000 free software downloads
http://DetachPipe.com - send huge email attachments

Bernard Toplak
Posts: 5
Joined: Fri Jun 16, 2006 12:13 am
Location: Varazdin, Croatia
Contact:

Postby Bernard Toplak » Tue Jun 27, 2006 9:33 am

Great news, Simon! :D ... I will gladly test it on those hex files I have.

Mail on the way! :wink:


Return to “TextPipe Tips and Tricks, Questions and Support”

Who is online

Users browsing this forum: No registered users and 1 guest