Objectif Lune joins Upland Software.Learn more >
March 2nd, 2022
OL Connect Workflow can only run a single configuration file. This means that if you have multiple solutions or projects that need to run on the same server, you need a way to integrate those into a single Workflow configuration before deploying. For the online samples hosted by Objectif Lune (e.g. Accounts Receivable, Proof of Delivery and others), we have solved this by standardizing the way these solutions are created so they can be combined easily. This still involves a manual process, but, when done right, it allows a straightforward deployment without having to change the actual solutions.
The following information applies to both PlanetPress Connect Workflow and PReS Connect Workflow, so this article refers to them collectively as Workflow.
Every solution or project should be organized in the same way. Usually, there are two categories of files: configuration files that don’t change at runtime, and runtime files that do change. We recommend having a strict separation between these categories. Store all static configuration files in a folder structure (the Solutions folder), and have a separate folder structure for all runtime files (the Workspace folder). Our overall folder structure looks like this:
\OL Connect Solutions\
\OL Connect Workspace\
Inside each project, we also have a standardized structure:
If a project has other configurations beyond the OL Connect resources and Workflow config (for instance, we use WordPress when a web front-end is needed), then these will sit in separate folders in the Configurations\ folder. Static resources that are required by configurations sit one level higher (sample data for instance).
To allow the sharing of static resources used by multiple projects, we store them in the “OL Connect Solutions\Common” folder. This can include scripts, OL Connect resources, JSON parameter files, etc.
OL Connect Solutions\Common
Anything that needs manual integration to allow multiple projects to co-exist, is put in the “OL Connect Solutions\Integration” folder. The main thing that can be found here, is the combined Workflow config with all projects in it, and perhaps the starting point for that integrated config, for instance if a common startup process is needed, or if there are global variables that have a scope beyond a single project.
OL Connect Solutions\Integration
If the individual projects need those “global globals” also to be able to run (try to avoid this, if possible), then these variables can exist in the individual project’s Workflow configs as well.
To maintain a strict distinction between static resources and runtime data, anything for which a static resource is the starting point, but that can then change at runtime, needs to be deployed automatically from the solutions folder into the workspace at startup. For instance, the global startup process could check for the presence of a parameters file in the workspace; if it’s missing, it could then copy the static file from the solutions folder to the workspace.
Anything specific to a DTAP environment should be stored in external resources. This allows promoting configurations through DTAP without having to make any change on each of the DTAP systems (which would defeat the purpose of having DTAP). For our solution samples, we use a JSON file that contains settings for each DTAP system:
"description": "Production server",
"description": "Test server",
"description": "Dev server",
At startup, this file is read and the right settings are selected based on the hostname. We can have as many different environments as we like with this setup. So, the startup process automatically adjusts the current system’s settings according to the parameters found in that configuration file, without any user intervention.
Anything that is global to a Workflow config, needs to get a project specific prefix, to avoid clashes when projects are combined. It helps if that prefix is short. For instance, we choose “ar” and “pod” for our samples. This does not only apply to Global Variables, but also to process names, the “action” in NodeJS Server Input tasks, subprocess names, OL Connect resources such as templates, data mappings, etc. Doing so is helpful as it allows you to visually group related items in the Workflow Configuration tool.
Processes, subprocesses, and global variables should always be in groups (it helps to use the name space prefix for the top level group). Global variables and subprocesses only have one grouping level, so when you have many of those, you can have multiple groups per project, and then you should use the name space prefix for each group.
Starting with OL Connect 2018.2, multiple startup processes are allowed, and process groups can be nested (groups within groups). This means that every project can have its own startup process (if it needs one), and its possible to create groups of processes within a project.
If there are any processes or variables that have a broader scope than a single project, these should go into a separate Workflow configuration. This config should go into a special “integration project”.
Whenever a project needs some kind of change, the change is done on the project’s Workflow config, and that config is then modified and tested in isolation. Once the changes are done, and everything seems to work in the development environment, there are two options:
\OL Connect Solutions\project X\
To have multiple projects run on the same server, we combine each individual project’s Workflow config into a single integrated Worfklow config. This is done in the Integration project on the development environment.
It is important to do this in the development environment, because it involves manual changes, and these should travel through the entire DTAP process.
When all projects have been set up as above, creating the integrated Workflow config should be the only thing that is necessary.
Let’s assume there is a Workflow config with some processes and/or variables that are only needed when running multiple projects in a single Connect Workflow service. This would exist in the “\OL Connect Solutions\Integration\Configurations\Workflow\”, and perhaps be named base-integrated.OL-workflow.
\OL Connect Solutions\Integration\Configurations\Workflow\
To create the integrated Workflow configuration, the following needs to happen:
The integrated-solutions.OL-Workflow can now be sent to the local Workflow service on the development server for testing. Once it’s successful, everything can be promoted to the test server.
If all solutions follow the rules laid out above, creating the integrated Workflow configuration can be done quickly and without risk.
When projects are set up as described above, promoting changes from Development to Test, Acceptance, and Production becomes easy:
Auto deploying projects, and externalizing environment specific parameters is essential to making this work smoothly.
Instead of promoting changes by copying files, it’s also an option to use a revision control system to manage your DTAP environment.
For our sample solution development, we use a revision control system to facilitate working with multiple people. Besides the obvious added control to manage changes, we also use this to further streamline our DTAP environment (DTP in our case, because we don’t use Acceptance). We use Git for this, but the same can be done with Subversion, or any other modern revision control system.
Our procedure for working with Git :
It’s a is pretty simple process that suits our needs as a small team.
While testing, we often troubleshoot and debug directly on the Test server, but the easiest way to apply fixes, is to fix them in development, commit those on the master branch, and then merge again to test. If changes are committed to the test branch, merging them from test to master could be more tricky than vice versa (with Git, that is).
If anything is wrong in Production, it’s possible to pull the prod branch to a different environment so we can troubleshoot without disrupting Production. Small emergency fixes or work-arounds can be tested in Test, and directly applied in Production if needed, while a longer term solution can be prepared in Development.
It’s worth mentioning that we are actively working on integrating revision control into the OL Connect suite of applications. Keep up to date with the latest changes by following this blog!
Tagged in: DTAP, Integration, Projects, Revision control, Versioning
Receive exclusive OL products news, tips and resources.
Your email address will not be published. Required fields are marked *
Notify me of followup comments via e-mail. You can also subscribe without commenting.