Subtle bug in Convert Numeric HTML/XML Entities to text

Get help with installation and running here.

Moderators: DataMystic Support, Moderators

Post Reply
DFH
Posts: 830
Joined: Sun Dec 09, 2007 2:49 am
Location: UK

Subtle bug in Convert Numeric HTML/XML Entities to text

Post by DFH » Mon Dec 10, 2018 4:28 am

Hi Simon,

I have just encountered some very rare errors in the output of the filter Convert Numeric HTML/XML Entities to text.

My input file contains 31,286 NCRs that are to be converted.
These are scattered through out a UTF-8 text file that has 36,888 lines and a total of 7,219,971 characters.

The vast majority of the NCRs are converted correctly.

However, there were 5 locations where the conversion was incorrect.

I can only conclude that there's a subtle software bug in TextPipe Standard 10.7.2.

The problem is in understanding the root cause, because the same NCRs are converted correctly in other locations in the file.

The errors were detected by comparing the output file with one obtained by using BabelPad version 1.0.0.4 to Convert NCRs to Unicode.

This issue is critical, seeing as the TextPipe conversion is producing these rare but wrong results.

The attached .diff file was generated using WinMerge. Not ideal, but it does provide the context.

NB. I can readily send you a copy of the input file by email so that you might investigate further in detail.

Best regards,

David
Attachments
Reina1569.Bible.UTF-8.diff.zip
Zip contains .diff generated by WinMerge 0.86
(950 Bytes) Downloaded 16 times

DFH
Posts: 830
Joined: Sun Dec 09, 2007 2:49 am
Location: UK

Re: Subtle bug in Convert Numeric HTML/XML Entities to text

Post by DFH » Mon Dec 10, 2018 4:44 am

Since posting the issue, I have found a workaround.

If the filter Convert Numeric HTML/XML Entities to text is plac1ed under a Restrict to each line in turn filter, the output file is without error.

I thereby conclude that the bug must have some relation to data caching or memory management.

David

DFH
Posts: 830
Joined: Sun Dec 09, 2007 2:49 am
Location: UK

Re: Subtle bug in Convert Numeric HTML/XML Entities to text

Post by DFH » Tue Feb 05, 2019 10:29 pm

A workaround is one thing, but were you able to confirm whether or not my conjecture about data caching was valid?

If so, is there anything that can be done to prevent the bad output that I observed without such a workaround?

David

Post Reply