Concurrent Processing Framework

Requirements

The CPF requires a database to store the CPF configuration, Job requests and Job results. The following databases are currently supported.

• PostgreSQL 9.1+
• Oracle 10g, 11g, or 12c

A developer may also require additional databases for use by their plug-in. They must deliver all required SQL scripts and instructions on how to install these databases.

NOTE: For PostgreSQL the server instance can be shared with other applications but it is recommended to have a separate “database” within that instance for CPF. For Oracle there is no such restriction.

Download SQL Scripts

The SQL scripts to install the database can be downloaded from the https://github.com/bcgov/cpf repository.

Use the following scripts will download the CPF Oracle and PostgreSQL scripts.

** UNIX/Mac**

svn co https://github.com/bcgov/cpf/trunk/sql
cd sql

Windows

svn co https://github.com/bcgov/cpf/trunk/sql
cd sql

NOTE: If you have previously downloaded the SQL use the following command from the sql directory to ensure that you have the latest version.

svn up

CPF Database Install Configuration

The database install scripts use the db.properties configuration file for database connection and configuration parameters. Copy the db-sample.properties file from the postgresql or oracle directory to use as a template.

NOTE: Change permissions on the db.properties so that only you have read/write permissions on the file to keep the passwords secret.

Edit the db.properties file.

For PostgreSQL, to avoid needing to enter in the passwords for each SQL command create a ~/.pgpass on UNIX or %APPDATA%\postgresql\pgpass.conf file on Windows. Set the permissions so that only you can read/write the pgpass.conf file. The file can be deleted after installation if required. The file should look something like this.

localhost:5432:*:postgres:postgres
localhost:5432:*:cpf:cpf_0wn3r

CPF Database and schema install

The CPF installation process will create the following database objects. If the objects already exist the script will either ignore that step or in the case of CPF tables it will prompt to confirm that the existing tables should be deleted.

• Create a cpf database for PostgreSQL. An existing Oracle database must already exist.
• Create a CPF tablespace to store the CPF data.
• Create a CPF_WEB_PROXY database role that will have CRUD permission on the tables.
• Create a CPF database user account that this the owner of all the CPF tables. NOTE: This account must not be used for any other purpose than managing the table definitions.
• Create a PROXY_CPF_WEB user account that is used by the CPF web application to access the database.
• Create a CPF schema for all of the CPF tables, sequences and indexes.
• Create all the CPF tables, sequences and indexes and grant appropriate permissions for these tables.

NOTE: For Oracle you will need to know the password for the SYSTEM database account, you will be prompted to enter this in the script.

Unix/Mac

DB_VENDOR={postgresql|oracle}
cd cpf/sql/${DB_VENDOR}
./install.sh

Windows

set DB_VENDOR={postgresql|oracle}
cd  cpf\sql\%DB_VENDOR%
install.cmd

During the installation script you may be prompted for the following information.

• Passwords for the SYSTEM or postgres user accounts.
• If the database already exists the following prompt will be displayed. Entering YES will delete all the existing data and create new tables.
  WARN: Do you want to drop the existing database including all data (YES/NO)?

NOTE: Contact the CPF developer, if there are any errors while running the script that are NOT related to file permissions or unknown user account passwords.

Create CPF directories

The CPF requires directories to be created on the server. The following directories must be created.

NOTE: This assumes the CPF home directory is /apps/cpf. Modify the commands and configuration below if a different directory is used.

Create the directories using the following commands. Make sure the directory permissions are set as shown in the table above.

UNIX/Mac

mkdir -p /apps/cpf
mkdir -p /apps/cpf/config
mkdir -p /apps/cpf/log
mkdir -p /apps/cpf/repository

Windows

md \apps\cpf
md \apps\cpf\config
md \apps\cpf\log
md \apps\cpf\repository

Create a CPF web application project

The CPF application can be configured to connect to different types of database and be configured or extended in other ways. Therefore instead of delivering a pre-packaged war file a maven project is created for each installation that contains the configuration for that environment.

Create the maven project using the following maven archetype commands. Replace any parameters with the correct values for your environment.

NOTE: Java 1.8.0 and Maven 3.3+ must be install. JAVA_HOME and M2_HOME must be set and the bin directories from both must be in the PATH.

UNIX/Mac

CPF_VERSION=5.0.2-SNAPSHOT
cd ~/projects
mvn \
  archetype:generate \
  -DinteractiveMode=false \
  -DarchetypeGroupId=ca.bc.gov.open.cpf \
  -DarchetypeArtifactId=cpf-archetype-web \
  -DarchetypeVersion=${CPF_VERSION} \
  -DgroupId=com.mycompany \
  -DartifactId=cpf* \
  -Dversion=1.0.0-SNAPSHOT \
  -DmodulePrefix=cpf \
  -DdatabaseVendor=postgresql \
  -DdatabasePassword=c0ncurr3n7 \
  -DworkerPassword=cpf_w0rk3r \
  -DcpfLogDirectory=/apps/cpf/log \
  -DcpfDirectoryUrl=file:///apps/cpf \
  -DmavenCacheDirectoryUrl=file:///apps/cpf/repository

Windows

set CPF_VERSION=5.0.2-SNAPSHOT
cd %HOMEDRIVE%%HOMEPATH%\projects
mvn ^
  archetype:generate ^
  -DinteractiveMode=false ^
  -DarchetypeGroupId=ca.bc.gov.open.cpf ^
  -DarchetypeArtifactId=cpf-archetype-web ^
  -DarchetypeVersion=%CPF_VERSION% ^
  -DgroupId=com.mycompany ^
  -DartifactId=cpf ^
  -Dversion=1.0.0-SNAPSHOT ^
  -DmodulePrefix=cpf ^
  -DdatabaseVendor=postgresql ^
  -DdatabasePassword=c0ncurr3n7 ^
  -DworkerPassword=cpf_w0rk3r ^
  -DcpfLogDirectory=C:/apps/cpf/log ^
  -DcpfDirectoryUrl=file:///C:/apps/cpf ^
  -DmavenCacheDirectoryUrl=file:///C:/apps/cpf/repository

NOTE: Windows and Unix require commands to be entered on a single line. The \ or ^ character are line continuation character that treats multiple lines as a single line. Therefore you can cut and paste the above text into a command window.

The following directory structure would be created if the command were run using the parameters above.

NOTE: Developers shouldn’t need to edit any of these configuration files. They are populated using the parameters specified in the maven archetype. If you need to change the cpf directory location or the database vendor it’s probably easier to delete the project and create a new project using the maven archetype. All database configurations are done using runtime configuration files and plug-in configuration is stored in the database.

Maven Settings Configuration

A profile must be created in the ~/.m2/settings.xml for each server environment that the CPF components are deployed to. The following example shows a full settings file with a single profile for the localhost server.

If you want to deploy the application to multiple servers you can create a profile for each server in your ~/.m2/settings.xml. The profile id should be the name of the server to deploy to. For example the following shows a profile for the localhost.

<settings>
  <profile>
    <id>localhost</id>
    <properties>
      <!-- Include the following for Tomcat deployment -->
      <tomcatManagerUrl>http://localhost:8080/manager/text</tomcatManagerUrl>
      <tomcatManagerUsername>admin</tomcatManagerUsername>
      <tomcatManagerPassword>...</tomcatManagerPassword>
    </properties>
  </profile>
</settings>

CPF Runtime Configuration

Any configuration that changes from one server to another is included in a runtime configuration file in the /apps/cpf/config directory on the server. No environment specific configuration is included in the compiled WAR files, i.e. the same war could be deployed to delivery, test and production environments.

Copy the sample-config/cpf.properties file to the /apps/cpf/config/cpf.properties directory on the server.

External Maven Repository Configuration

If the CPF is not deployed on the development server then the worker will need to be configured to point to the maven respositories that the CPF plug-ins are deployed to.

Edit the following configuration file. You will need to re-build and deploy if this file is modified.

~/projects/cpf/cpf.app/src/main/resources/cpf-web-properties.sf.xml

Add the following entries to the cpfWebProperties map. The first entry must be modified to the value used for the mavenCacheDirectory specified above. The second entry value must be the local file URL or remote http URL to a shared maven repository that you will deploy the plug-ins to. See Maven Deploy Plug-in for deploying to a maven repository.

<entry
  key="mavenRepository.root"
  value="file:///apps/cpf/repository/"
/>

<entry key="mavenRepository.repositoryLocations">
  <list>
    <value>http://mycompany.com/maven/repository/</value>
  </list>
</entry>

Deploy to Tomcat 8

The plug-in project web services & scheduler war and worker war files can be deployed to a Tomcat 8 server.

Use the following command to compile and deploy to Tomcat.

mvn -P tomcat8Deploy,localhost clean install

If you created multiple profiles use the profile name of the server you wish to deploy to.