New filter proposal: Repeat Filter(s)

Get help with installation and running here.

Moderators: DataMystic Support, Moderators

alnico
Posts: 74
Joined: Fri Oct 12, 2007 11:57 pm

New filter proposal: Repeat Filter(s)

Postby alnico » Mon Feb 14, 2011 12:41 am

I am not sure if this is possible with the way TextPipe processes...but here goes...

Since it is possible to create nice structured sub filter processes (trees) in TextPiple...
What if it were possible to insert a parent filter called "Repeat Filter(s)" that simply relies on Vbscript "Do Until Loop" (in essence it would dynamically wrap the tree with a Do Until loop).
It's parameters would be: Repeat until: [macro] [operator] [macro/number].

I know there is many ways to loop...but having one simple way up front would be a great enhancement.

Possible? Problems with my logic ;-)

User avatar
DataMystic Support
Site Admin
Posts: 2154
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: New filter proposal: Repeat Filter(s)

Postby DataMystic Support » Mon Feb 14, 2011 8:05 am

Hi - this is actually possible already by using subfilters of a VBScript filter. The VBscript filters controls when and how the subfilters get applied. See the help file for more details.
Regards,

Simon Carter, http://DataMystic.com/forums/index.php
http://PredictBGL.com - Insulin dose calculator for Type 1 diabetes
http://DownloadPipe.com - 250,000 free software downloads
http://DetachPipe.com - send huge email attachments

alnico
Posts: 74
Joined: Fri Oct 12, 2007 11:57 pm

Re: New filter proposal: Repeat Filter(s)

Postby alnico » Tue Feb 15, 2011 12:44 am

Ya, I figured modularizing and bringing this out of the programming environment (like all other filters) would benefit non-programmers and be quick and easy to set up.

User avatar
DataMystic Support
Site Admin
Posts: 2154
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: New filter proposal: Repeat Filter(s)

Postby DataMystic Support » Tue Feb 15, 2011 8:21 am

I understand your logic, but then making the filter able to cope with all the possible variations of loop start and end conditions etc - better off just delegating this to a custom script.

How did you plan to use this filter?
Regards,

Simon Carter, http://DataMystic.com/forums/index.php
http://PredictBGL.com - Insulin dose calculator for Type 1 diabetes
http://DownloadPipe.com - 250,000 free software downloads
http://DetachPipe.com - send huge email attachments

alnico
Posts: 74
Joined: Fri Oct 12, 2007 11:57 pm

Re: New filter proposal: Repeat Filter(s)

Postby alnico » Wed Feb 16, 2011 2:58 am

Basically this is what I need:

|--Repeat Filter(s) (Do Until Loop)
| |--Find (Replace first only in turn)
| | |--Transform
| |--Sort
|--Loop

Even using the VBScript filter…I am not convinced this could be done. As I can switch sub-filters off/on, but I cannot repeat a filter when “Replace first only” is active.
TextPipe as a line processor combined with VBScript gets me a little confused…as I am always in a restriction ;-)

Thinking about this more…

Maybe the Repeat Filter(s) suggestion is not needed if it were possible to solve the same problem by adding checkbox option:
“Replace first in turn” to the “Perl pattern” filter.

Thus its implementation could be like this:

|--Find (Replace first in turn)
| |--Transform
|--Sort

This would repeat all filters below the 'Find' on the same tree level in turn until no more matches are found (does have the potential for an endless loop however, maybe there's a way to control it?).
To control the loop...can add it to a restriction.

User avatar
DataMystic Support
Site Admin
Posts: 2154
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: New filter proposal: Repeat Filter(s)

Postby DataMystic Support » Wed Feb 16, 2011 7:31 am

Thanks, but can I please see a practical example of where this would be applied?
Regards,

Simon Carter, http://DataMystic.com/forums/index.php
http://PredictBGL.com - Insulin dose calculator for Type 1 diabetes
http://DownloadPipe.com - 250,000 free software downloads
http://DetachPipe.com - send huge email attachments

alnico
Posts: 74
Joined: Fri Oct 12, 2007 11:57 pm

Re: New filter proposal: Repeat Filter(s)

Postby alnico » Thu Feb 17, 2011 1:17 am

I am normalizing html tables that contain the attribute: rowspan (duplicating and positioning x times).

Each table row, cell is given a multipart line ID...such as 001.001.001
Each rowspan line is duplicated x times and line ID incremented using VBScript
Then the entire <table> must be sorted
Then line IDs must be reset using VBScript
Then repeat the process (loop) for the next rowspan found.

The problem is that TextPipe can only do work within restrictions all at once...even using variables and 'add section header', etc wont work...because always limited by the restriction.

Here is an example table where the first rowspan is duplicated 3 times and line ID updated.
If we now sort this it would be correct...but with subsequent rowspans they sometimes will be one position off (that is because we are duplicating all the rowspan's at once and then one sort (it needs to loop so that line IDs may be reset).

I have this all working, but of course the looping part is missing.
This is a very complicated process...the model I have designed works...but just getting TextPipe to do it is challenging ;-)


Code: Select all

<table width="500" border="1">
001.001.001   <tr>
001.002.001   <td rowspan="3">a</td>
002.002.000   <td rowspan="3">a</td>
003.002.000   <td rowspan="3">a</td>
001.003.001   <td rowspan="4">c</td>
001.004.001   <td>d</td>
001.005.001   </tr>
002.001.001   <tr>
002.002.001   <td>f</td>
002.003.001   </tr>
003.001.001   <tr>
003.002.001   <td>h</td>
003.003.001   </tr>
004.001.001   <tr>
004.002.001   <td>i</td>
004.003.001   <td>j</td>
004.004.001   </tr>
005.001.001   <tr>
005.002.001   <td>k</td>
005.003.001   <td>l</td>
005.004.001   <td rowspan="3">m</td>
005.005.001   </tr>
006.001.001   <tr>
006.002.001   <td>n</td>
006.003.001   <td>o</td>
006.004.001   </tr>
007.001.001   <tr>
007.002.001   <td>p</td>
007.003.001   <td>q</td>
007.004.001   </tr>
</table>

User avatar
DataMystic Support
Site Admin
Posts: 2154
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: New filter proposal: Repeat Filter(s)

Postby DataMystic Support » Fri Feb 18, 2011 8:11 am

Ok, I am confused - why are you doing this? Are you trying to sort the table by ColA, ColB, ColC?
Regards,

Simon Carter, http://DataMystic.com/forums/index.php
http://PredictBGL.com - Insulin dose calculator for Type 1 diabetes
http://DownloadPipe.com - 250,000 free software downloads
http://DetachPipe.com - send huge email attachments

alnico
Posts: 74
Joined: Fri Oct 12, 2007 11:57 pm

Re: New filter proposal: Repeat Filter(s)

Postby alnico » Sat Feb 19, 2011 1:21 am

Sorry,

Here is what exactly I am trying to accomplish. http://www.andrewjwelch.com/code/xslt/table/table-normalization.html

I currently have to do part of my processing in TextPipe then with 3rd party app (without command line) then bring the data back into TextPipe to continue.
It is a messy process that I am trying to simplify.

User avatar
DataMystic Support
Site Admin
Posts: 2154
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: New filter proposal: Repeat Filter(s)

Postby DataMystic Support » Mon Feb 21, 2011 11:11 am

Ok, that's messy. I still don't know why you would want to do this - surely there are very few situations where you want to repeat table cells, unless they are &nbsp;'s or blank (or a fixed background color).
I still maintain the looping is highly unnatural for text stream processing.

Anyway:
but I cannot repeat a filter when “Replace first only” is active.

When you call subFilterEntireText the entire text you pass to the subfilters is treated as one file - so Replace First Only works as it would for a file. If you don't want it to work, then you could use

Code: Select all

initSubFilters()
subFilterChunk( inputString : string )
subFilterChunk( inputString : string )
subFilterChunk( inputString : string )
...
flushSubfilters()


repeatedly.
Regards,

Simon Carter, http://DataMystic.com/forums/index.php
http://PredictBGL.com - Insulin dose calculator for Type 1 diabetes
http://DownloadPipe.com - 250,000 free software downloads
http://DetachPipe.com - send huge email attachments

alnico
Posts: 74
Joined: Fri Oct 12, 2007 11:57 pm

Re: New filter proposal: Repeat Filter(s)

Postby alnico » Fri Feb 25, 2011 1:48 am

Thanks Simon,

I just got this 'normalization' process figured out by running an external process and bringing it back into TextPipe...no small feat ;-)

You ask "why I would want to do this"...imagine exporting html tables from print catalog layout software (with merged cells) and preparing it for database import. Tables/data MUST be normalized.

Thanks,
Brent

User avatar
DataMystic Support
Site Admin
Posts: 2154
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: New filter proposal: Repeat Filter(s)

Postby DataMystic Support » Fri Feb 25, 2011 7:36 am

Glad to hear it!

Thanks - now I understand the why!
Regards,

Simon Carter, http://DataMystic.com/forums/index.php
http://PredictBGL.com - Insulin dose calculator for Type 1 diabetes
http://DownloadPipe.com - 250,000 free software downloads
http://DetachPipe.com - send huge email attachments


Return to “TextPipe Tips and Tricks, Questions and Support”

Who is online

Users browsing this forum: No registered users and 2 guests