Migration Troubleshooting

This section regroups the problems you can encounter when you migrate a P6 Core instance to a different version.

Warning

This guide is not complete, so contact the R&D team if something goes wrong during the migration.

Version 6.0.0-alpha-2

Cluster Removed from DSL

In order to simplify the use of the transaction DSL, the notion of cluster was removed. The value of cluster parameters passed to lower level layers is now hardcoded to B2BOX. Please remove the first parameter when calling transaction.buildPK().

Version 6.0.0-alpha-3

One key pair per application publisher is needed to sign an application package, therefore the format of the application profile file (.app) has changed. Old ones can no longer be used, ask the R&D team to generate new ones for you. In the future, you will be able to create your .app files in the P6 Management Console.

Also, the format of the application package has changed and (it’s compressed). Therefore old applications can no longer be imported and need to be re-packaged.

Version 6.0.0-alpha-4

Old Database Tables Removed

In the database, the following tables were deleted:

  • universe
  • role
  • menu
  • routingrule
  • transformer

Please update your stored procedures accordingly.

Libraries Upgraded

This should not have an impact unless you rely on specific features in your scripts:

  • Apache POI for manipulating Microsoft Office documents was upgraded to version 4.0.1.
  • Apache Camel was upgraded to version 2.23.1.

App Profile File Simplified

Application profile files were simplified. You need to re-generate you .app files.

B2GUID Removed

If you use B2GUID in your scripts, please use java.util.UUID instead.

B2RequiredEndpointScope Removed

The B2RequiredEndpointScope annotation was removed, it’s been deprecated for a long time and you can safely delete it from your scripts.

message DSL Binding Removed

Please use transaction in your scripts instead.

_DEFAULT_ Variable Replaced

When working with smart documents, the _DEFAULT_ pipeline variable was renamed to text.

Version 6.0.0-alpha-5

Before the migration

1/ We integrate in this version some declarative table partitioning. That feature is only available in PostgreSQL 11.

The database version needs to be upgraded from 9.6.1 to 11.3. The upgrade procedure is explained here https://www.postgresql.org/docs/11/upgrading.html

If you use Platform 6 Install, follow these steps and the migration will be done for you:

  • Pull the latest version of the project in your instance folder.
  • While your instance is still running on the old database, run the script pgsql_export.sh. Your dumps are stored in a folder called database_dumps.
  • Stop your instance by running stop_platform6.sh and edit your .env file, set PGSQL_VERSION=11.3.
  • Import you data by running pgsql_import.sh, that’s it! Then you can delete the database dump files if you wish.

2/ The database schema was renamed from b2head to p6core. Renaming is handled dynamically by our migration scripts (including the migration of all stored procedures). But you still need to apply manually the following statements BEFORE starting the migration:

DROP TABLE b2head.flyway_schema_history;
ALTER SCHEMA b2head RENAME TO p6core;

3/ You need to ensure that the cluster column of the Item table only contains the following values:

  • .work
  • .crossreferencing
  • B2BOX

You can fetch these values by running the following query:

SELECT DISTINCT cluster FROM p6core.item;

Warning

All other values will be deleted from item table

If you still have entries with the values .configuration or PROVISIONING, you can safely delete them because configuration is managed in table serviceconfig in P6 and old UI users are no longer used, by running this query:

DELETE FROM p6core.item WHERE cluster in ('.configuration', 'PROVISIONING');

4/ The number of key columns on the Item table has been decreased. The columns iid5 and iid6 were removed from this table. If you use these columns, please concatenate old values and store them using columns from iid1 to iid4 and rework your indexes accordingly.

5/ The views configuration are stored in the serviceconfig table. The only allowed values for the <DataCluster> xml node are:

  • .work
  • B2BOX

To list all values, launch this query:

SELECT xpath('//DataCluster/text()', xmlparse(document (content ->> 'xmlView')))
FROM p6core.serviceconfig
WHERE id3 = 'views';

If you come across any other value, make sure to replace it with the value ‘.work’ as shown below:

Migration example: 

UPDATE p6core.serviceconfig
SET content = replace(content::varchar, '<DataCluster>DummyValue</DataCluster>', '<DataCluster>.work</DataCluster>')::jsonb
WHERE id3='views';

To migrate from a DummyValue to the .work cluster.

6/ Indexes

Please take a look at the indexes defined on the item table and remove all the ones you think are no longer used. The automatic migration process will try to upgrade the remaining indexes. Deleting indexes frees disk space and most importantly speeds up migration and reduces chances of failure.

Migration duration: Configuration: 2 CPU / 6 GiB memory

300k entries on item table takes ~25min 850k entries on item table takes ~1h 15min 3,5M entries on item table takes ~3h

Durations may vary depending on the environment and the configuration.

6/ Configuration update

In the application configuration file application.conf, service.workflow.datamodel property has been renamed to service.workflowsteps.datamodel and the default value has changed from MessageInfo to TransactionInfo.

7/ Instance Base Context and Swagger(OpenApi)

The Instance record created via the OAuth Settings portal menu requires an update.

The Base context of a p6 instance is now: http//hostname:8080/p6 (not http://hostname:8080/b2box/rest/resource)

Swagger support has been upgraded to V3 which is now a standard know as OpenAPI. The V1 Swagger user interface is no longer embedded in the product. The latest Swagger user interface can be downloaded from https://swagger.io/tools/swagger-ui/

The Swagger UI should use the endpoint: http://hostname:8080/p6/openapi

Note

At this stage, you can finally start your instance in version 6.0.0-alpha-5. Change the version in the Docker Compose file and start your instance.

DSL

fileFromUri DSL Variable Removed

Please use fileFromUrl from the uri DSL

DSL access

To access to the DSL you need to prefix them by p6., for example:

log.debug p6.counter.list()

Transaction DSL

This version makes TransactionInfo the default model for transactions rather than MessageInfo. Also, in order to ease P6 learning curve, the term ‘project’ was replaced with ‘save’, therefore the following functions were renamed:

  • putMI -> putTI
  • projectMI -> saveTI which validates against a TransactionInfo and not a MessageInfo
  • project -> save
  • p6projectAndRouteMI -> saveAndRouteTI which validates against a TransactionInfo and not a MessageInfo
  • p6projectAndRoute -> saveAndRoute
  • p6route -> route which is not to be confused with the old function route which used routing rules!

In P6 version 6.0.0 all support for the old routing rules was removed, hence the following functions disappeared:

  • projectAndRouteMI
  • projectAndRoute
  • route

DataModel

The autoloaded data model WFWorkItem has been renamed to WorkflowTask. Make sure to update your stored procedures, XSLT resources, scripts…

Integrations

Each Platform 6 instance should now be defined and configured as an Integration using the new Administration panel.

See: Adding Your Instance ‘Integration’

Routes

Web3j

The component is no longer called P6Web3j, because we now rely on the standard Web3j Camel component. Therefore, all routes with endpoint URI p6web3j://... should be renamed to web3j://....

The producer query parameter transactionIndex was renamed to indexto comply with the official Web3j Camel component.

Observable operations were renamed to align with Web3j version 4.3:

  • ETH_LOG_OBSERVABLE -> ETH_LOG_FLOWABLE
  • ETH_JSON_LOG_OBSERVABLE -> ETH_JSON_LOG_FLOWABLE
  • ETH_BLOCK_HASH_OBSERVABLE -> ETH_BLOCK_HASH_FLOWABLE
  • ETH_PENDING_TRANSACTION_HASH_OBSERVABLE -> ETH_PENDING_TRANSACTION_HASH_FLOWABLE
  • TRANSACTION_OBSERVABLE -> TRANSACTION_FLOWABLE
  • PENDING_TRANSACTION_OBSERVABLE -> PENDING_TRANSACTION_FLOWABLE
  • BLOCK_OBSERVABLE -> BLOCK_FLOWABLE
  • REPLAY_BLOCKS_OBSERVABLE -> REPLAY_PAST_BLOCKS_FLOWABLE
  • REPLAY_TRANSACTIONS_OBSERVABLE -> REPLAY_PAST_TRANSACTIONS_FLOWABLE
  • CATCH_UP_TO_LATEST_BLOCK_OBSERVABLE -> REPLAY_ALL_PAST_BLOCKS_FLOWABLE
  • CATCH_UP_TO_LATEST_TRANSACTION_OBSERVABLE -> REPLAY_ALL_PAST_TRANSACTIONS_FLOWABLE
  • CATCH_UP_TO_LATEST_AND_SUBSCRIBE_TO_NEW_BLOCKS_OBSERVABLE -> REPLAY_PAST_AND_FUTURE_BLOCKS_FLOWABLE
  • CATCH_UP_TO_LATEST_AND_SUBSCRIBE_TO_NEW_TRANSACTIONS_OBSERVABLE -> REPLAY_PAST_AND_FUTURE_TRANSACTIONS_FLOWABLE

Pipeline Variables

The following automatically filled pipeline variables were renamed: - platform6.request.cluster -> platform6.request.dataPartition - platform6.request.concept -> platform6.request.dataType

Routing Order Attribute Changes

Routing orders were designed to store calls to adapters and their parameters. This was later extended to also include service calls via the svc:\\ service URL syntax.

Since adapters have been completely removed in Platform 6, the Routing Order attributes: Adapter and Adapter Parameters have been re-purposed to Service and Service Parameters

Warning

The svc:\\ URL syntax is no longer supported.

The Service attribute must now contain the name of the target service. For example platform6.scripts (or scripts for short)

The Service Parameters attribute now contains a comma separated list of URL encoded name/value pairs

The change to format and use of attributes means that any routing orders created prior to this release will not be able to be reprocessed ( Unless they are first heavily edited first using the Routing Orders UI.)

Windows support

Installing Platform 6 natively on Windows is no longer supported. If you want to use Platform 6 on Windows it has to be done using Docker through the Platform 6 install scripts on Github.

Dependency Upgrades

  • Bouncy Castle from 1.59 to 1.61
  • Flying Saucer PDF from 9.0.7 to 9.1.18
  • Groovy from 2.5.6 to 2.5.7
  • Web3j from 3.5.0 to 4.3.1

log4j.properties

If you have made a copy of the log4j.properties file and stored it as an override in your instance p6core_data/conf folder you must update it.

The line:

log4j.appender.b2box = io.platform6.core.impl.logging.B2boxAppender

should be changed to

log4j.appender.platform6 = io.platform6.core.impl.logging.Platform6Appender

And the corresponding reference at the top of the file to appender b2box must be changed to platform6

log4j.rootCategory=ALL, platform6, rfile

Version 6.0.0-beta-1

ApplicationId vs. InstanceId

A terminology change during development of Platform 6 led to a confusing mix of terms where applicationId and instanceId where used interchangeably to refer the same thing!

We have now standardised on InstanceId

Warning

This effects a mandatory attribute in the application.conf of every instance

The application.conf attribute:

applicationid = myp6instance

must be changed to:

instance.id = myp6instance

before Platform 6 will start.

DSL

The package name containing the DSL has been renamed from com.amalto.b2box.groovy to io.platform6.core.dsl and the DSL are no longer accessible without using the prefix p6..

Annotations

The following annotations was renamed: - B2GroovyBinding -> P6GroovyBinding - B2ExecutionContext -> P6ExecutionContext - B2User -> P6User

and also moved from the package com.amalto.b2box.groovy.dsl.bindingto io.platform6.core.impl.application.scripting.annotations

Transaction DSL

The getSerialForm method has been removed

Table DSL

The following function was renamed: - upsertRecords -> upsert

Note: Existing scripts will be updated to use the new DSL function so no manual action is required.

Warning: If you are using P6Sync don’t forget to retrieve the online modification using the -pull option

Utils DSL

The following bindings has been moved (for both scripts and routes): - escapeXml -> p6.utils.escapeXml - unescapeXml -> p6.utils.unescapeXml - pause -> p6.utils.pause

Note: Existing scripts will be updated so no manual action is required.

Warning: If you are using P6Sync don’t forget to retrieve the online modification using the -pull option

Routes

The following bindings has been moved for route definition: - utils -> p6.utils - configuration -> p6.configuration - camel -> p6.camel - service -> p6.service - appconfig -> p6.appconfig

Note: Existing routes will be updated so no manual action is required.

Warning: If you are using P6Sync don’t forget to retrieve the online modification using the -pull option

Exception

The use of io.platform6.core.api.B2boxException has been replaced by io.platform6.common.util.P6Exception.

Note: Existing routes will be updated to use the new exception so no manual action is required.

Warning: If you are using P6Sync don’t forget to retrieve the online modification using the -pull option

Configuration files

The following file was renamed: - b2boxcontext.xml -> p6context.xml

The configuration entries starting with b2audit. have been renamed to audit.. And configuration entries starting with b2auth. have been renamed to p6auth..

Log appender renamed

io.platform6.core.impl.logging.B2boxAppender was renamed to io.platform6.core.impl.logging.Platform6Appender.

Services

Transactions

The endpoint [DELETE] /apis/v2/platform6.transactions/transaction/{view}/{ids:.*} has been removed. You should use the endpoint [DELETE] /apis/v2/platform6.transactions/transactions instead.

Endpoint permissions

The endpoints to deal with file submission have change and you need the following permissions:

Service createSession, addFile deleteFile, deleteSession listFiles
transactions [transactions=edit] [transactions=delete] [transactions=read]
bundledresources [bundledresources=edit] [bundledresources=delete] [bundledresources=read]
workflowsteps [workflowsteps=edit] [workflowsteps=delete] [workflowsteps=read]

Version 6.0.0-beta-2

Endpoint permissions

Endpoint permissions have been simplified and the ones that have been modified are listed bellow.

REST API Old permissions New permissions
Add an item SERVICE_ID=read && SERVICE_ID=edit SERVICE_ID=edit
Delete an item SERVICE_ID=read && SERVICE_ID=edit SERVICE_ID=read && SERVICE_ID=delete
Delete items SERVICE_ID=read && SERVICE_ID=edit SERVICE_ID=read && SERVICE_ID=delete
Export items SERVICE_ID=read && SERVICE_ID=edit SERVICE_ID=read

Applications

The following services will be automatically restarted following application installation and un-installation (assuming the application contains service items deployed against each service)

  • Workflow Steps
  • Routes
  • Transactions

No manual action is required, except cleaning up your install and uninstall scripts in your applications from calls that restart the above services.

Configuration

All the following configuration keys have been replaced by p6.realm:

  • groovy.b2.realm
  • default.p6.realm
  • b2box.b2.realm
  • service.workflow.b2.realm

The following configuration entries have been renamed:

  • counters.* -> p6.service.counters.*
  • service.workflow.* -> p6.service.workflowsteps.*
  • service.* -> p6.service.*
  • com.amalto.service.<name.serviceName>.limiter.* -> p6.service.<name>.limiter.*
  • io.platform6.core.impl.application.camel.p6log.LogConsumer -> p6.camel.p6log.LogConsumer
  • audit.* -> p6.service.auditing.*
  • store.* -> p6.store.*
  • swagger.ui -> p6.swagger.ui
  • io.platform6.core.impl.platform.jobcontrol.JobControlManagerImpl.* -> p6.job.*
  • job.control.execution.messages.length -> p6.job.control.execution.messages.length
  • disable.get.cache.headers -> p6.cache.annotation
  • cmb.service.runner.pool.size.<name> -> p6.service.<name>.runner.pool.size

DSL Changes

The method defaultTransactionManager in the ethereumrpc DSL was renamed to noOpTransactionManager.

The DSL p6.control was renamed to p6.script. The method moduleNames in the script DSL was renamed to list.

Note

The renaming is automatically handled in scripts, no manual action is required.

The method getCompiled in the resource DSL was removed.

Version 6.0.0

Before the migration

The default redirect URI for OAuth clients was changed from http://www.b2box.com to http://www.platform6.io. Therefore if your instance still uses the default b2box client to connect to P6 Auth, add the following line to application.conf:

p6auth.client.redirect.uri="http://www.b2box.com"

Variables

The following variables have been renamed in the application: - B2BOX_DATA -> P6_DATA - B2BOX_HOME -> P6_HOME

This affects: - Scripts (main script and resources) - Routes (template and script segments) - URL (any use of an URL starting with file:) - Bundled resources (target root) - Workflow steps (URL inside the definition of thr xml step)

Note

The renaming is automatically handled in the previous cases by SQL migration scripts

Use official Camel Web3j routes

This release is aligned with the official Camel Web3j module. Therefore many custom headers that were added in previous versions were removed. Here is the list of removed headers:

  • logAddress
  • logBlockHash
  • logBlockNumber
  • logData
  • logIndex
  • logTransactionHash
  • logTransactionIndex
  • logType
  • logTopics

These headers are the properties of a org.web3j.protocol.core.methods.response.Log that were extracted. To migrate your blockchain listener routes, you need to serialize every incoming Log from the blockchain to JSON, then parse the JSON string and read the property you want. A full example of such a route is provided in the Demo App.

Version 6.0.1

DSL Changes

After all enhancements made to the AppConfig DSL, the Ethereum DSL no longer relies on the Tables DSL to store credentials. Therefore, the following functions were removed:

  • String addNewCredentials(String tableName, Map keys, String fieldName, String password, boolean useFullScrypt)
  • Credentials getCredentials(String tableName, Map keys, String fieldName, String password)

Some Changes To Defaults

  • The default p6auth.client.id in reference.conf is no longer b2box. It is now p6core. You may need to change you current Client definition using OAuth setting or switch to the more current Integrations way of integrating p6core instances with P6 Auth.
  • The default log file name has changed in log4j.properties from b2box.log to p6core.log.
  • The default install location of the product within the docker container has changed from /opt/b2box to /opt/p6core.
  • The default TMP folder used within the docker container has changed from /tmp/b2box to /tmp/p6core.

New JVM

For licencing reasons and for future independence we have stopped using the Oracle distribution of Java. We now use the official OpenJDK provided Docker container image:

  • openjdk:8-jre-alpine

This is a significant change. Not only are we moving away from the Oracle distribution, but we are also moving from a JDK to a JRE. In combination with the Alpine linux distribution the overall size of the P6 Core container has been significantly reduced.

Note: We no longer extend the original b2base image that contained the S6 service manager. We no longer configure the postfix service or run P6 Core as a service.

Services

Bundled resources

Endpoints

The endpoint [DELETE] /apis/v2/platform6.bundledresources/bundledresource/{name}/{appKey: .*} has been removed. You should use the endpoint [POST] /apis/v2/platform6.bundledresources/bundledresources/delete instead.

The endpoint [PUT] /apis/v2/platform6.bundledresources/bundledresource has been removed. You should use the endpoint [PUT] /apis/v2/platform6.bundledresources/bundledresources/{sessionToken: .*} instead. The body of the request has to be the BundledResource item and the sessionToken can be empty.

The endpoint [POST] /apis/v2/platform6.bundledresources/bundledresource has been removed. You should use the endpoint [POST] /apis/v2/platform6.bundledresources/bundledresources/{sessionToken: .*} instead. The body of the request has to be the BundledResource item and the sessionToken can be empty.

Version 6.0.3

Before the migration

All class packages were renamed from com.amalto.b2box... to io.platform6.... Therefore make sure class names are correct in your properties files in the conf folder. For instance, com.amalto.b2box.core.store.impl.dialect.pgsqlxml.mappers.proc.ReportingDataSourceProcMapper was renamed to io.platform6.core.store.impl.dialect.pgsqlxml.mappers.proc.ReportingDataSourceProcMapper.

Pipeline Changes

The content type of variables injected in the pipeline changed as follows:

  • application/b2box.itempk -> application/p6core.itempk
  • application/b2box.dom -> application/p6core.dom
  • application/b2box.i18n -> application/p6core.i18n

Note

Renaming is automatically handled by SQL migration scripts, no manual action required.