Upland OL User community

Removing Lines in Preprocessor Script

#1

Hi,
I have a question about preprocessors. Can a preprocessor be used to remove whole lines from a text input.

I have been given text data where there almost always 8 header lines which appear at the beginning of the file, but occasionally a file is supplied that only has six. Usually it is as below
image

The difference is lines 7 and 8. Is it possible to remove those using a script in the preprocessor?
Thanks
Chris

#2

Of course it is.

The following script replaces CRLF in the data by <br> character.

var reader = openTextReader(data.filename,"UTF-8"); 
var tmpFile = createTmpFile(); 
var writer = openTextWriter(tmpFile.getPath(),"UTF-8"); 
var line = null; 
while((line = reader.readLine()) != null){ 
	line = line.replace(/(?:\\r\\n)/g, "<br>"); 
	line = line.replace(/(?:\\n)/g, "<br>"); 
	writer.write(line); 
} 
writer.close(); 
reader.close(); 

deleteFile(data.filename); 
tmpFile.move(data.filename);

You can adapt it to remove your undesired lines like so:

var reader = openTextReader(data.filename,"UTF-8"); 
var tmpFile = createTmpFile(); 
var writer = openTextWriter(tmpFile.getPath(),"UTF-8"); 
var line = null; 
while((line = reader.readLine()) != null){ 
	if(!(line === ':O:0' || line === ':S:0')) writer.write(line); 
} 
writer.close(); 
reader.close(); 

deleteFile(data.filename); 
tmpFile.move(data.filename);
#3

That’s great thank you very much. I had to adjust it slightly as it removed all the line breaks. updated script below

var reader = openTextReader(data.filename);
var tmpFile = createTmpFile();
var writer = openTextWriter(tmpFile.getPath());
var line = null;
while((line = reader.readLine()) != null){
if(!(line === ‘:o:0’ || line === ‘:S:0’)) writer.write((line)+"\n");
}
writer.close();
reader.close();

deleteFile(data.filename);
tmpFile.move(data.filename);