OL Learn

james123456 · July 12, 2022, 11:19am

Hi PlanetPress wiz,

Hope you are all okay and well.

I have a data that can contain up-to and not excessive of 1000 records. But for some random reason an output record gets generated and it’s a record from this data file as text. This happens randomly, but has started to become more frequent. When I re-process this file, I get my output and is processed correctly as what I want.

The data file is being split into chunks then processing these chunks are processed one at a time. The setup in the workflow is really straight forward and easy. The process has a self-replication process ticked and set so 5%. I have had issues with this in previous jobs, so I tend to keep it at between 5%-10%. This is so when I try to do something like extracted data and writing into a text file

Is anyone having the same issue as me? And if so any work-arround for this any chance?

Thank you in advance.

Marten · July 14, 2022, 9:34am

Hi @james123456,

I have a data that can contain up-to and not excessive of 1000 records. But for some random reason an output record gets generated and it’s a record from this data file as text

I assume that you have a input file (CSV?) which contains 1,000 records and that you sometimes have a output file which contains data of a single record as text instead of the expected output, can you please confirm if this is correct?

Were you able to check the PlanetPress Watch logging of such a moment? If so, do you see anything strange in the logging – For example: Certain Workflow plugins has been skipped because a Condition is true where it should be false – or any error or warning messages?

james123456 · July 15, 2022, 9:03am

Hi @Marten,

Thank you for your reply. It made me think a bit more

The data file is text that is delimited with a FF (FormFeed) character.
Looking at the logs on the Datamapperengine, it looks like there are some warning exeptions being generated “The step is out of bound (DME000130)” step(s) being out of bounds. Though a generated PDF file is on disk. This is okay.

Where there is an Error message with “com.objectiflune.datamining.internal.exceptions.EStepFailed: The step is out of bound (DME000130)” and I think this is when the text file is just output as text.

I have noticed this only happen when I run the file through the workflow. If I run it through the designer via an output creation so that all records are created, all file are generated okay.

There is also the “ERROR [main] com.objectiflune.contentcreation.html.mergeengine.impl.a.raiseError(HtmlMergeCoreUtils.java) uncaught exception: out of memory.
com.objectiflune.layoutengine.html.embed.GeckoJSRuntimeError: uncaught exception: out of memory”

Followed by “ERROR [IPC Input] com.objectiflune.contentcreation.html.mergeengine.impl.HtmlMergeToPDF.exportToPdf(HtmlMergeToPDF.java) Error during content creation (CC3000023)
org.eclipse.core.runtime.CoreException: uncaught exception: out of memory.”

The above logs are from the “Mergeengine log file” and no errors in the Datamapperengine or Weaverengine log files. I guese the Merger engines need more memory.

I will keep testing/changing somethings to correctly indentify the issue.

Thanks again.

TDGreer · July 15, 2022, 4:12pm

Connect (incorrectly, in my never humble opinion) interprets “line termination” characters as “new line” characters. This will cause a spurious extra, empty, record at the end of your job. This MIGHT be the source of your issue.

You can override this behavior by checking the “Ignore CR/LF/FF at end of file” checkbox on the Settings tab of the Data Map.

james123456 · July 18, 2022, 8:31am

Hi @TDGreer, I have the check box ticked as I thought this initially. I will have a look and do some twicks on the Mergeengine as this is where the error originates. Thanks for your advice.

james123456 · August 3, 2022, 3:57pm

Hi,

The only thing I can find is this below from the mergeengine logs file.
Any ides what the issue could be? There is enough memory to process the file but I don’t know what the issue is. Maybe somene here in the forum can give me some pointers.

[2022-08-03 16:29:52,887] ERROR [main] com.objectiflune.contentcreation.html.mergeengine.impl.HtmlMergeMasterPageJSCallback.function(HtmlMergeMasterPageJSCallback.java) Error during on-demand merge of masterpage: org.eclipse.core.runtime.CoreException: C:\Users\xxxxxxxxxxx\AppData\Local\Temp\TZCBUWfW.HTM.part could not be saved, because the source file could not be read.Â¶Â¶Try again later, or contact the server administrator. (CC3000018)>>>line endings sanitized<<< (Core_6250)
org.eclipse.core.runtime.CoreException: C:\Users\xxxxxxxxxxx\AppData\Local\Temp\TZCBUWfW.HTM.part could not be saved, because the source file could not be read.

Try again later, or contact the server administrator.
	at com.objectiflune.contentcreation.html.mergeengine.impl.internal.HtmlMergeCoreUtils.callInUIThread(HtmlMergeCoreUtils.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.internal.HtmlMergeCoreUtils.runInUIThread(HtmlMergeCoreUtils.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.b.loadResources(b.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.b.getUnit(b.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.q.merge(q.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.HtmlMergeMasterPageJSCallback.function(HtmlMergeMasterPageJSCallback.java)
	at com.objectiflune.layoutengine.html.embed.e.callJava(e.java)
	at org.mozilla.xpcom.internal.XPCOMJavaProxy.callXPCOMMethod(Native Method)
	at org.mozilla.xpcom.internal.XPCOMJavaProxy.invoke(XPCOMJavaProxy.java:201)
	at com.sun.proxy.$Proxy97.oL_evaluateJS(Unknown Source)
	at com.objectiflune.layoutengine.html.embed.u.a(u.java)
	at com.objectiflune.layoutengine.html.embed.u.evaluate(u.java)
	at com.objectiflune.layoutengine.html.embed.Browser.evaluate(Browser.java)
	at com.objectiflune.layoutengine.html.MozillaBrowser.a(MozillaBrowser.java)
	at com.objectiflune.layoutengine.html.MozillaBrowser.evaluate(MozillaBrowser.java)
	at com.objectiflune.layoutengine.html.pagination.j.run(j.java)
	at com.objectiflune.layoutengine.html.pagination.PaginationProvider.a(PaginationProvider.java)
	at com.objectiflune.layoutengine.html.pagination.PaginationProvider.paginate(PaginationProvider.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.p.run(p.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.internal.HtmlMergeCoreUtils.runAndWait(HtmlMergeCoreUtils.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.a.b(a.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.a.mergeAndPaginate(a.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.HtmlMergeEngine.mergeAndPaginate(HtmlMergeEngine.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.HtmlMergeEngine.mergeAndPaginate(HtmlMergeEngine.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.HtmlMergeToPDF.lambda$1(HtmlMergeToPDF.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.internal.HtmlMergeCoreUtils.lambda$0(HtmlMergeCoreUtils.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.internal.a.run(a.java)
	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:40)
	at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:185)
	at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4001)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3629)
	at com.objectiflune.contentcreation.mergeengine.product.internal.MergeEngineApplication.start(MergeEngineApplication.java)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:203)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:401)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:255)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:653)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:590)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1461)
	at org.eclipse.equinox.launcher.Main.main(Main.java:1434)
	
	
	[2022-08-03 16:29:52,896] ERROR [main] com.objectiflune.contentcreation.html.mergeengine.impl.a.raiseError(HtmlMergeCoreUtils.java) Error: org.eclipse.core.runtime.CoreException: C:\Users\xxxxxxxxxxx\AppData\Local\Temp\TZCBUWfW.HTM.part could not be saved, because the source file could not be read.Â¶Â¶Try again later, or contact the server administrator.>>>line endings sanitized<<< (Core_6250)
com.objectiflune.layoutengine.html.embed.GeckoJSRuntimeError: Error: org.eclipse.core.runtime.CoreException: C:\Users\xxxxxxxxxxx\AppData\Local\Temp\TZCBUWfW.HTM.part could not be saved, because the source file could not be read.

Try again later, or contact the server administrator.
	at com.objectiflune.layoutengine.html.js2java.JavascriptErrorHandler.onJavascriptError(JavascriptErrorHandler.java)
	at com.objectiflune.layoutengine.html.js2java.JavascriptErrorCallback.function(JavascriptErrorCallback.java)
	at com.objectiflune.layoutengine.html.embed.e.callJava(e.java)
	at org.mozilla.xpcom.internal.XPCOMJavaProxy.callXPCOMMethod(Native Method)
	at org.mozilla.xpcom.internal.XPCOMJavaProxy.invoke(XPCOMJavaProxy.java:201)
	at com.sun.proxy.$Proxy97.oL_evaluateJS(Unknown Source)
	at com.objectiflune.layoutengine.html.embed.u.a(u.java)
	at com.objectiflune.layoutengine.html.embed.u.evaluate(u.java)
	at com.objectiflune.layoutengine.html.embed.Browser.evaluate(Browser.java)
	at com.objectiflune.layoutengine.html.MozillaBrowser.a(MozillaBrowser.java)
	at com.objectiflune.layoutengine.html.MozillaBrowser.evaluate(MozillaBrowser.java)
	at com.objectiflune.layoutengine.html.pagination.j.run(j.java)
	at com.objectiflune.layoutengine.html.pagination.PaginationProvider.a(PaginationProvider.java)
	at com.objectiflune.layoutengine.html.pagination.PaginationProvider.paginate(PaginationProvider.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.p.run(p.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.internal.HtmlMergeCoreUtils.runAndWait(HtmlMergeCoreUtils.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.a.b(a.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.a.mergeAndPaginate(a.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.HtmlMergeEngine.mergeAndPaginate(HtmlMergeEngine.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.HtmlMergeEngine.mergeAndPaginate(HtmlMergeEngine.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.HtmlMergeToPDF.lambda$1(HtmlMergeToPDF.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.internal.HtmlMergeCoreUtils.lambda$0(HtmlMergeCoreUtils.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.internal.a.run(a.java)
	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:40)
	at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:185)
	at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4001)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3629)
	at com.objectiflune.contentcreation.mergeengine.product.internal.MergeEngineApplication.start(MergeEngineApplication.java)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:203)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:401)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:255)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:653)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:590)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1461)
	at org.eclipse.equinox.launcher.Main.main(Main.java:1434)
	
	[2022-08-03 16:29:52,902] ERROR [IPC Input] com.objectiflune.contentcreation.html.mergeengine.impl.HtmlMergeToPDF.exportToPdf(HtmlMergeToPDF.java) Error during content creation (CC3000023)
org.eclipse.core.runtime.CoreException: Error: org.eclipse.core.runtime.CoreException: C:\Users\xxxxxxxxxxx\AppData\Local\Temp\TZCBUWfW.HTM.part could not be saved, because the source file could not be read.

Try again later, or contact the server administrator.
	at com.objectiflune.contentcreation.html.mergeengine.impl.internal.HtmlMergeCoreUtils.callInUIThread(HtmlMergeCoreUtils.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.internal.HtmlMergeCoreUtils.runInUIThread(HtmlMergeCoreUtils.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.HtmlMergeToPDF.exportToPdf(HtmlMergeToPDF.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.HtmlMergeEngine.a(HtmlMergeEngine.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.HtmlMergeEngine.createContent(HtmlMergeEngine.java)
	at com.objectiflune.contentcreation.mergeengine.product.internal.MergeEngineCommandProcessor.createContent(MergeEngineCommandProcessor.java)
	at com.objectiflune.contentcreation.mergeengine.product.internal.f.run(f.java)
	at com.objectiflune.contentcreation.mergeengine.product.internal.MergeEngineCommandProcessor.a(MergeEngineCommandProcessor.java)
	at com.objectiflune.contentcreation.mergeengine.product.internal.MergeEngineCommandProcessor.a(MergeEngineCommandProcessor.java)
	at com.objectiflune.contentcreation.mergeengine.product.internal.MergeEngineCommandProcessor.processMessages(MergeEngineCommandProcessor.java)
	at com.objectiflune.contentcreation.mergeengine.product.MergeEngineThread.instantiateCommandProcessor(MergeEngineThread.java)
	at com.objectiflune.core.ipc.EngineThread.run(EngineThread.java)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: com.objectiflune.layoutengine.html.embed.GeckoJSRuntimeError: Error: org.eclipse.core.runtime.CoreException: C:\Users\xxxxxxxxxxx\AppData\Local\Temp\TZCBUWfW.HTM.part could not be saved, because the source file could not be read.

Try again later, or contact the server administrator.
	at com.objectiflune.layoutengine.html.js2java.JavascriptErrorHandler.onJavascriptError(JavascriptErrorHandler.java)
	at com.objectiflune.layoutengine.html.js2java.JavascriptErrorCallback.function(JavascriptErrorCallback.java)
	at com.objectiflune.layoutengine.html.embed.e.callJava(e.java)
	at org.mozilla.xpcom.internal.XPCOMJavaProxy.callXPCOMMethod(Native Method)
	at org.mozilla.xpcom.internal.XPCOMJavaProxy.invoke(XPCOMJavaProxy.java:201)
	at com.sun.proxy.$Proxy97.oL_evaluateJS(Unknown Source)
	at com.objectiflune.layoutengine.html.embed.u.a(u.java)
	at com.objectiflune.layoutengine.html.embed.u.evaluate(u.java)
	at com.objectiflune.layoutengine.html.embed.Browser.evaluate(Browser.java)
	at com.objectiflune.layoutengine.html.MozillaBrowser.a(MozillaBrowser.java)
	at com.objectiflune.layoutengine.html.MozillaBrowser.evaluate(MozillaBrowser.java)
	at com.objectiflune.layoutengine.html.pagination.j.run(j.java)
	at com.objectiflune.layoutengine.html.pagination.PaginationProvider.a(PaginationProvider.java)
	at com.objectiflune.layoutengine.html.pagination.PaginationProvider.paginate(PaginationProvider.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.p.run(p.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.internal.HtmlMergeCoreUtils.runAndWait(HtmlMergeCoreUtils.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.a.b(a.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.a.mergeAndPaginate(a.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.HtmlMergeEngine.mergeAndPaginate(HtmlMergeEngine.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.HtmlMergeEngine.mergeAndPaginate(HtmlMergeEngine.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.HtmlMergeToPDF.lambda$1(HtmlMergeToPDF.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.internal.HtmlMergeCoreUtils.lambda$0(HtmlMergeCoreUtils.java)
	at com.objectiflune.contentcreation.html.mergeengine.impl.internal.a.run(a.java)
	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:40)
	at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:185)
	at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4001)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3629)
	at com.objectiflune.contentcreation.mergeengine.product.internal.MergeEngineApplication.start(MergeEngineApplication.java)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:203)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:401)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:255)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:653)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:590)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1461)
	at org.eclipse.equinox.launcher.Main.main(Main.java:1434)
	
	
	
	
	[2022-08-03 16:29:52,939] ERROR [IPC Input] com.objectiflune.contentcreation.html.mergeengine.impl.HtmlMergeToPDF.exportToPdf(HtmlMergeToPDF.java) org.eclipse.core.runtime.CoreException: uncaught exception: out of memory
	
	
	
	
	
	[2022-08-03 16:29:52,939] ERROR [IPC Input] com.objectiflune.contentcreation.mergeengine.product.internal.MergeEngineCommandProcessor.createContent(MergeEngineCommandProcessor.java) Error: org.eclipse.core.runtime.CoreException: C:\Users\xxxxxxxxxxx\AppData\Local\Temp\TZCBUWfW.HTM.part could not be saved, because the source file could not be read.Â¶Â¶Try again later, or contact the server administrator.>>>line endings sanitized<<< (Core_6250)

Marten · August 4, 2022, 7:54am

Hi @james123456,

Can you let me know please if you were able to find out what the memory usage of the Merge Engine was the moment the reported incident occurred and if it was equal to or higher than the value set via the Connect Server Configuration application? In version 2022.1 of PReS Connect this is set via: OL Connect Server Configuration application > (Preferences window) > Engines > Memory > Merge Engine (MB).

james123456 · August 4, 2022, 8:14am

Hi @Marten,

We currently had the memory lower than this as shown in the image and only 1 Data Converter. So ended up bumping it to 2. It’s way more than enough I think! Hope this helps.

It’s just driving me insane. When these files are output as text, when I drop them back in to process there is no issue. There is no condition in the workflow that is being missed or anything. I am now thinking of prrocessing the whole file in an output crreation instead. As it’s the only solution at this moment.

Workflow Processing jobs, some random data generated as text process?