2019-06-23

16: Optimally Use LibreOffice as a Files Converter (a Java Implementation)

<The previous article in this series | The table of contents of this series | The next article in this series>

Between OpenDocument/Microsoft Office/etc. formats and to the PDF/CSV format, fast and versatile with the document possibly tweaked. A workable sample.

Topics


About: UNO (Universal Network Objects)
About: LibreOffice
About: Apache OpenOffice
About: the Java programming language

The table of contents of this article


Starting Context



Target Context


  • The reader will understand how to implement file conversion directly using the UNO API in Java, and will have a workable sample.

Orientation


There is an article on how to build any sample program of this series.

There is an article on how to create a LibreOffice or Apache OpenOffice daemon.

There is an article on how to create a LibreOffice or Apache OpenOffice Microsoft Windows service.

There is an article on encrypting an Office file with any opening password with UNO.

There is an article on setting any editing password into an OpenDocument file with UNO.

There is an article on setting any editing password into a binary Word/Excel file with UNO.

There is an article on setting any editing password into an Office Open XML file with UNO.

There is an article on setting the size of any page of any word processor document with UNO.

There is an article on setting the size of any page of any spread sheets document with UNO.

There is an article on setting any page property of any word processor document with UNO.

There is an article on setting any page property of any spread sheets document with UNO.

There is an article on exporting any office document into a PDF file as per full specifications with UNO.

There is an article on writing any spread sheet to a CSV file in any format with UNO.

There is an article on writing all the spread sheets of any spread sheets document to CSV files with UNO.


Main Body

Stage Direction
Here are Special-Student-7, Douglas (a Java programmer), and Renee (a Java programmer) in front of a computer.


0: A Note: 'soffice --convert-to' Will Be Never Used


Special-Student-7
While it is clearly stated that the concept part of this article is supposed to have been read, I can guess that a goodly portion of the readers have not and will not read it, and considering the fact that many people are fixated on using 'soffice --convert-to', I feel it appropriate to remind you again that 'soffice --convert-to' will be never used here.

The reason is elaborated in the concept part.


1: An Implementation of the File Conversion Logic with the UNO API


Special-Student-7
Supposing that you already know what you need to know from the concept part of this article, let us immediately see a Java implementation of the file conversion logic with the UNO API, where 'l_underlyingRemoteUnoObjectsContextInXComponentContext' is a UNO objects context to the LibreOffice or Apache OpenOffice instance, 'l_originalFileUrl' is the URL of the original file, 'l_targetFileUrl' is the URL of the target file, and 'l_fileStoringFilterName' is the file-storing-filter name.

@Java Source Code
package theBiasPlanet.unoUtilitiesTests.fileConvertingTest1;

import java.util.regex.Matcher;
import java.util.regex.Pattern;
import com.sun.star.beans.PropertyState;
import com.sun.star.beans.PropertyValue;
import com.sun.star.frame.XDispatchProvider;
import com.sun.star.frame.XStorable2;
import com.sun.star.frame.XSynchronousDispatch;
import com.sun.star.uno.AnyConverter;
import com.sun.star.uno.UnoRuntime;
~
import com.sun.star.uno.XInterface;
import com.sun.star.util.XCloseable;
~

public class Test1Test {
	private static Pattern c_urlRegularExpression;
	
	static {
		 c_urlRegularExpression = Pattern.compile ("(.*:)(.*)");
	}
	
	// the 'com.sun.star.util.URLTransformer' UNO service can be used if you want to, as I do this manually.
	private static com.sun.star.util.URL getUrlInURL (String a_url) {
		com.sun.star.util.URL l_urlInURL = new com.sun.star.util.URL ();
		l_urlInURL.Complete = a_url;
		l_urlInURL.Main = l_urlInURL.Complete;
		Matcher l_regularExpressionMatcher = c_urlRegularExpression.matcher (l_urlInURL.Complete);
		if (l_regularExpressionMatcher != null && l_regularExpressionMatcher.lookingAt ()) {
			l_urlInURL.Protocol = l_regularExpressionMatcher.group (1);
			l_urlInURL.User = "";
			l_urlInURL.Password = "";
			l_urlInURL.Server = "";
			l_urlInURL.Port = 0;
			l_urlInURL.Path = l_regularExpressionMatcher.group (2);
			l_urlInURL.Name = "";
			l_urlInURL.Arguments = "";
			l_urlInURL.Mark = "";
		}
		return l_urlInURL;
	}
	
	public static void main (String [] a_argumentsArray) {
		~
				String l_originalFileUrl = a_argumentsArray [1];
				String l_targetFileUrl = a_argumentsArray [2];
				String l_fileStoringFilterName = a_argumentsArray [3];
				
				XDispatchProvider l_underlyingUnoDesktopInXDispatchProvider = UnoRuntime.queryInterface (XDispatchProvider.class, (XInterface) l_underlyingRemoteUnoObjectsContextInXComponentContext.getValueByName ("/singletons/com.sun.star.frame.theDesktop"));
				XSynchronousDispatch l_underlyingFileOpeningUnoDispatcherInXSynchronousDispatch = UnoRuntime.queryInterface (XSynchronousDispatch.class, l_underlyingUnoDesktopInXDispatchProvider.queryDispatch (getUrlInURL ("file:///"), "_blank", -1));
				
				com.sun.star.util.URL l_originalFileUrlInURL = getUrlInURL (l_originalFileUrl);
				PropertyValue [] l_unoDocumentOpeningPropertiesArray = new PropertyValue [4];
				l_unoDocumentOpeningPropertiesArray [0] = new PropertyValue ("ReadOnly", -1, Boolean.valueOf (true), PropertyState.DIRECT_VALUE);
				l_unoDocumentOpeningPropertiesArray [1] = new PropertyValue ("Hidden", -1, Boolean.valueOf (true), PropertyState.DIRECT_VALUE);
				l_unoDocumentOpeningPropertiesArray [2] = new PropertyValue ("OpenNewView", -1, Boolean.valueOf (true), PropertyState.DIRECT_VALUE);
				l_unoDocumentOpeningPropertiesArray [3] = new PropertyValue ("Silent", -1, Boolean.valueOf (true), PropertyState.DIRECT_VALUE);
				PropertyValue [] l_unoDocumentStoringPropertiesArray = new PropertyValue [2];
				l_unoDocumentStoringPropertiesArray [0] = new PropertyValue ("FilterName", -1, l_fileStoringFilterName, PropertyState.DIRECT_VALUE);
				l_unoDocumentStoringPropertiesArray [1] = new PropertyValue ("Overwrite", -1, Boolean.valueOf (true), PropertyState.DIRECT_VALUE);
				
				com.sun.star.uno.Any l_underlyingOriginalUnoDocumentInAny = (com.sun.star.uno.Any) (l_underlyingFileOpeningUnoDispatcherInXSynchronousDispatch.dispatchWithReturnValue (l_originalFileUrlInURL, l_unoDocumentOpeningPropertiesArray));
				boolean l_hasSucceeded = false;
				if (! (AnyConverter.isVoid (l_underlyingOriginalUnoDocumentInAny))) {
					XStorable2 l_underlyingOriginalUnoDocumentInXStorable2 = UnoRuntime.queryInterface (XStorable2.class, (XInterface) l_underlyingOriginalUnoDocumentInAny.getObject ());
					try {
						l_underlyingOriginalUnoDocumentInXStorable2.storeToURL (l_targetFileUrl, l_unoDocumentStoringPropertiesArray);
						l_hasSucceeded = true;
					}
					catch (Exception l_exception) {
						throw l_exception;
					}
					UnoRuntime.queryInterface (XCloseable.class, l_underlyingOriginalUnoDocumentInXStorable2).close (false); 
				}
				else {
					System.out.println (String.format ("### The original file: '%s' cannot be opened.", l_originalFileUrl));
				}
		~
	}
}


Douglas
. . . Well, does that mean that I can just copy it into my code and my program already begins to convert files?

Special-Student-7
If your code has already gotten a UNO objects context and has appropriately set the URLs and the file-storing-filter name, yes, your code will have satisfied the minimum requirements.

Douglas
A "UNO objects con . . ."? What the hell is that?

Special-Student-7
. . . You can get it according to an article (for just or for a scrupulous way).

Douglas
"scrupulous", you said?

Renee
Douglas does not even understand that concept.

Douglas
What are possible filter names?

Special-Student-7
They are cited in the concept part of this article.

Douglas
So, I just set a filter name, and the file will be stored in the corresponding format?

Special-Student-7
Yes.

Let me briefly comment on the code.

The code is doing exactly what I said in a section of the concept part of this article: open the original file, store the opened document in the target format, and close the opened document.

'dispatchWithReturnValue' is opening the original file, 'storeToURL' is storing the opened document, and 'close' is closing the opened document.

'l_underlyingFileOpeningUnoDispatcherInXSynchronousDispatch' is a UNO dispatcher and does not need to be gotten per file.


2: On Properties for Opening or Storing


Special-Student-7
As is seen in the above code, you can set some properties for opening the file. In fact, these are the specifications of the possible properties, where the datum types are Java types (instead of UNO types).

The nameThe datum typeThe description
ReadOnlyBooleanwhether the opened document cannot be written back into the original file: 'true'-> cannot, 'false'-> can
HiddenBooleanwhether the opened document is hidden: 'true'-> hidden, 'false'-> not hidden
OpenNewViewBooleanwhether the document is opened in a new view: 'true'-> in a new view, 'false'-> not in a new view
SilentBooleanwhether the document is opened silently: 'true'-> silently, 'false'-> not silently
PasswordStringthe opening password

A property you may be interested with is the opening password.

Renee
So, I can silently convert a file that is protected by the opening password.

Douglas
But I don't know the password; only the user knows it.

Special-Student-7
That means that you have to present an interface for the user to input the password, then your program will set the password there.

Renee
Douglas! Are you a child? You should be able to think that by your self!

Special-Student-7
Please note that UNO 'string' does not officially accept 'null', so, if there is no opening password, the property itself should not be specified, although 'null' might happen to be accepted.

And please note that 'ReadOnly' does not mean that the opened document cannot be modified, but means that the document cannot be written back into the original file, so, it does not need to be 'false' in order for the document to be tweaked and written into a new file.

You can set some properties also for storing the document. In fact, these are the specifications of the possible properties, where the datum types are Java types (instead of UNO types).

The nameThe datum typeThe description
FilterNameStringthe filter name
FilterDatadepends on the casethe filter-specific data for some filters: not always but often an array of 'com.sun.star.beans.PropertyValue'
FilterOptionsStringthe filter-specific data for some filters
AsTemplateBooleanwhether the document is stored as a template: 'true'-> as a template, 'false'-> not as a template
AuthorStringthe author name
DocumentTitleStringthe document title
EncryptionDatacom.sun.star.beans.NamedValue []the encryption data for some filters
PasswordStringthe opening password for some filters
CharacterSetStringthe characters set
VersionShortthe version number
CommentStringthe version description
OverwriteBooleanwhether the file is overwritten: 'true' -> overwritten, 'false' -> not overwritten
ComponentDatadepends on the casesome filter-specific data: not always but often an array of 'com.sun.star.beans.NamedValue'
ModifyPasswordInfodepends on the casethe editing password data for some filters

The opening password is really nothing but encryption data; the reason why there are both of 'EncryptionData' and 'Password' is that some filters use one and some other filters use the other; for details, please refer to another article.

The situation concerning setting an editing password is rather complicated: the editing password has to be hashed by a format-specific algorithm and the hash has to be set into a format-specific location, which is not necessarily inside the file-storing properties. . . . The details are explained in an article for OpenDocument files, for Microsoft binary files, and for Office Open XML files.

Douglas
. . . All I understand is that it is not easy.

Special-Student-7
As my workable sample has implemented the logic, you can use it, if you will.

Renee
What exactly can be set into those "filter-specific" properties like 'FilterData' and 'FilterOptions'?

Special-Student-7
That cannot be explained in this article for the general logic of file conversion: please refer to some other articles like for PDF files and for CSV files.


3: On Tweaking the Document


Special-Student-7
The target file may not be satisfactorily controlled only via the file-storing properties.

Then, you can tweak the opened document.

Douglas
"tweak"? How?

Special-Student-7
In the above code, 'l_underlyingOriginalUnoDocumentInXStorable2' represents the UNO object of the opened document, via which you can tweak the opened document.

Douglas
Being told "you can" . . .

Special-Student-7
That is of course document-specific and requirement-specific, and please refer to some other articles like for tweaking page properties of word processor documents (here and here) and for tweaking page properties of spread sheets documents (here and here).


4: Here Is a Workable Sample


Special-Student-7
A workable sample can be downloaded from here.

The sample program takes as an argument the path of a JSON file (in fact, it can take multiple JSON file paths) that contains file conversion orders, and fulfills the orders.

The specifications of the JSON file are like these.

@JSON Source Code
[
	{Disabled: %whether this record is disabled or not -> 'true' or 'false'%, 
	OriginalFileUrl: %the original file URL, in which any operating system environment variable can be used like '%{HOME}'%, 
	TargetFileUrl: %the target file URL, in which any operating system environment variable can be used like '%{HOME}'%, 
	FileStoringFilterName: %the file-storing-filter name%, 
	OpeningPassword: %the opening password for the original file%, 
	Title: %the target file title%, 
	EncryptingPassword: %the encrypting password for the target file%, 
	EditingPassword: %the editing password for the target file%, 
	CertificateIssuerName: %the certificate issuer name for signing the target file%, 
	CertificatePassword: %the certificate password for signing the target file%, 
	SignatureTimeStampAuthorityUrl: %the time stamp authority URL for signing the target file%, 
	DocumentTailorNames: [%a document tailor name%, . . .], 
	AllSpreadSheetsAreExported: %whether all the spread sheets are exported or not: 'true' or 'false'%, 
	TargetFileNamingRule: %the naming rule for target CSV files: '0'-> each CSV file is named with the sheet index, '1'-> each CSV file is named with the sheet name%, 
	CsvItemsDelimiterCharacterCode: %the character code of the CSV items delimiter: '44'-> ',' for example%, 
	CsvTextItemQuotationCharacterCode: %the character code of the CSV text item quotation: '34'-> '"' for example%, 
	CsvCharactersEncodingCode: %the CSV characters encoding code: '76' -> UTF-8, '65535' -> UCS-2, '65534' -> UCS-4, '11' -> US-ASCII, '69' -> EUC_JP, '64' -> SHIFT_JIS%, 
	CsvAllTextItemsAreQuoted: %whether all the CSV text items are quoted or not: 'true' or 'false'%, 
	CsvContentsAreExportedAsShown: %whether the CSV contents are exported as shown on the sheets: 'true' or 'false'%, 
	CsvFormulaeThemselvesAreExported: %whether the sheet cell formulae themselves are exported into the CSV files: 'true' or 'false'%, 
	HiddenSpreadSheetsAreExported: %whether also the hidden spread sheets are exported%}, 
	. . .
	{. . .}
]

Douglas
What the hell is "tailor"?

Special-Student-7
Any tailor is an object that is tasked with tweaking the document in a specific way.

Douglas
So, where can I find the one I want?

Special-Student-7
I have prepared a few sample tailors, but you will have to create one yourself if you have a specific need.

Douglas
Huh? Am I supposed to create one?

Renee
Yes, you are.

Anyway, am I supposed to specify the class name?

Special-Student-7
No, the object name: there may be some multiple tailors from a class, constructed with different parameters.

Douglas
JSON file doesn't work for me; my orders are as Web requests.

Special-Student-7
It is a sample, sir; you can modify it as you like; for example, you could form an in-memory JSON record from each Web request and pass the JSON record into the 'executeConversionOrder' method, which takes such a JSON record.

Douglas
Oh? So, I can use the method, as it is?

Special-Student-7
If the current JSON specifications are enough for you, yes, if you need additional capabilities in the JSON record, you can modify the 'executeConversionOrder' method together with the 'FileConversionOrderExtendedJsonDatumParseEventsHandler' JSON parse-events-handler.

Douglas
So, my immediate job is to construct the JSON record from each Web request . . .

Special-Student-7
I have made the sample to accept multiple orders files, which are processed concurrently, in order to demonstrate that conversions can be done concurrently in this optimal way.


4-1: Building the Sample Project


Special-Student-7
How to build the sample project is explained in an article.

Especially, note that the projects contain also implementations in some other programming languages, which you, a Java programmer, will probably not need; how to ignore such unnecessary code is explained there.

The main project is 'filesConverter'.


4-2: Executing the Sample Program


Special-Student-7
Before we execute the sample program, we have to have started a LibreOffice or Apache OpenOffice instance so that it accepts connections from clients, as we have learned how in a previous article.

Stage Direction
Special-Student-7 starts a LibreOffice instance with the port number 2002.

Special-Student-7
We need a conversion orders JSON file (in the specifications cited above), and we will use the sample JSON file included in the archive file as 'filesConverter/data/FileConversionOrders.json', but at least the file URLs inside it have to be adjusted.

We can execute the sample program like this with the current directory positioned at the sample project directory.

@cmd Source Code
gradle i_executeJarTask -Pi_mainClassName="theBiasPlanet.filesConverter.programs.FilesConverterConsoleProgram" -Pi_commandLineArguments="\"socket,host=localhost,port=2002,tcpNoDelay=1;urp;StarOffice.ComponentContext\" \"data/FileConversionOrders.json\""

Stage Direction
Special-Student-7 executes the command in the terminal with the current directory positioned at the sample project directory.

Special-Student-7
We can just change the host name from 'localhost' if the LibreOffice or Apache OpenOffice instance is in a remote host, if some firewalls do not block the communication, of course.

And you can specify multiple JSON files like this.

@cmd Source Code
gradle i_executeJarTask -Pi_mainClassName="theBiasPlanet.filesConverter.programs.FilesConverterConsoleProgram" -Pi_commandLineArguments="\"socket,host=localhost,port=2002,tcpNoDelay=1;urp;StarOffice.ComponentContext\" \"data/FileConversionOrders.json\" \"data/FileConversionOrders2.json\""


4-3: Understanding the Sample Code


Special-Student-7
Note that the sample code uses my utility projects ('coreUtilitiesToBeDisclosed' and 'unoUtilitiesToBeDisclosed'), which include code that is not directly related to the work of file conversion; please understand that it is not so easy to excise only irrelevant parts of the utility projects, because the code is intertwined.

The file conversion logic is in the 'convertFile' method (except for converting to CSV files) or the 'convertSpreadSheetsDocumentFileToCsvFiles' method (for converting to CSV files) of the 'theBiasPlanet.unoUtilities.filesConverting.FilesConverter' class, which (the method) takes the document tailors and the file-storing properties, among some other things.

In order to install a new document tailor, you have to have the class as a 'theBiasPlanet.unoUtilities.documentsHandling.UnoDocumentTailor' child and register an instance of the class with its unique name into the 'l_unoDocumentTailorNameToUnoDocumentTailorMap' variable of the 'theBiasPlanet.filesConverter.programs.FilesConverterConsoleProgram' class.


References


<The previous article in this series | The table of contents of this series | The next article in this series>