Skip to content

Migration Troubleshooting

This section regroups the problems you can encounter when you migrate a P6 Core instance to a different version.

Warning

This guide is not complete, so contact the R&D team if something goes wrong during the migration.

Version 6.0.0-alpha-2

Cluster Removed from DSL

In order to simplify the use of the transaction DSL, the notion of cluster was removed. The value of cluster parameters passed to lower level layers is now hardcoded to B2BOX. Please remove the first parameter when calling transaction.buildPK().

Version 6.0.0-alpha-3

One key pair per application publisher is needed to sign an application package, therefore the format of the application profile file (.app) has changed. Old ones can no longer be used, ask the R&D team to generate new ones for you. In the future, you will be able to create your .app files in the P6 Management Console.

Also, the format of the application package has changed and (it’s compressed). Therefore old applications can no longer be imported and need to be re-packaged.

Version 6.0.0-alpha-4

Old Database Tables Removed

In the database, the following tables were deleted:

  • universe
  • role
  • menu
  • routingrule
  • transformer

Please update your stored procedures accordingly.

Libraries Upgraded

This should not have an impact unless you rely on specific features in your scripts:

  • Apache POI for manipulating Microsoft Office documents was upgraded to version 4.0.1.
  • Apache Camel was upgraded to version 2.23.1.

App Profile File Simplified

Application profile files were simplified. You need to re-generate you .app files.

B2GUID Removed

If you use B2GUID in your scripts, please use java.util.UUID instead.

B2RequiredEndpointScope Removed

The B2RequiredEndpointScope annotation was removed, it’s been deprecated for a long time and you can safely delete it from your scripts.

message DSL Binding Removed

Please use transaction in your scripts instead.

_DEFAULT_ Variable Replaced

When working with smart documents, the _DEFAULT_ pipeline variable was renamed to text.

Version 6.0.0-alpha-5

Before the migration

1/ We integrate in this version some declarative table partitioning. That feature is only available in PostgreSQL 11.

The database version needs to be upgraded from 9.6.1 to 11.3. The upgrade procedure is explained here https://www.postgresql.org/docs/11/upgrading.html

If you use Platform 6 Install, follow these steps and the migration will be done for you:

  • Pull the latest version of the project in your instance folder.
  • While your instance is still running on the old database, run the script pgsql_export.sh. Your dumps are stored in a folder called database_dumps.
  • Stop your instance by running stop_platform6.sh and edit your .env file, set PGSQL_VERSION=11.3.
  • Import you data by running pgsql_import.sh, that’s it! Then you can delete the database dump files if you wish.

2/ The database schema was renamed from b2head to p6core. Renaming is handled dynamically by our migration scripts (including the migration of all stored procedures). But you still need to apply manually the following statements BEFORE starting the migration:

DROP TABLE b2head.flyway_schema_history;
ALTER SCHEMA b2head RENAME TO p6core;

3/ You need to ensure that the cluster column of the Item table only contains the following values:

  • .work
  • .crossreferencing
  • B2BOX

You can fetch these values by running the following query:

SELECT DISTINCT cluster FROM p6core.item;

Warning

All other values will be deleted from item table

If you still have entries with the values .configuration or PROVISIONING, you can safely delete them because configuration is managed in table serviceconfig in P6 and old UI users are no longer used, by running this query:

DELETE FROM p6core.item WHERE cluster in ('.configuration', 'PROVISIONING');

4/ The number of key columns on the Item table has been decreased. The columns iid5 and iid6 were removed from this table. If you use these columns, please concatenate old values and store them using columns from iid1 to iid4 and rework your indexes accordingly.

5/ The views configuration are stored in the serviceconfig table. The only allowed values for the <DataCluster> xml node are:

  • .work
  • B2BOX

To list all values, launch this query:

SELECT xpath('//DataCluster/text()', xmlparse(document (content ->> 'xmlView')))
FROM p6core.serviceconfig
WHERE id3 = 'views';

If you come across any other value, make sure to replace it with the value ‘.work’ as shown below:

Migration example: 

UPDATE p6core.serviceconfig
SET content = replace(content::varchar, '<DataCluster>DummyValue</DataCluster>', '<DataCluster>.work</DataCluster>')::jsonb
WHERE id3='views';

To migrate from a DummyValue to the .work cluster.

6/ Indexes

Please take a look at the indexes defined on the item table and remove all the ones you think are no longer used. The automatic migration process will try to upgrade the remaining indexes. Deleting indexes frees disk space and most importantly speeds up migration and reduces chances of failure.

Migration duration: Configuration: 2 CPU / 6 GiB memory

300k entries on item table takes ~25min 850k entries on item table takes ~1h 15min 3,5M entries on item table takes ~3h

Durations may vary depending on the environment and the configuration.

6/ Configuration update

In the application configuration file application.conf, service.workflow.datamodel property has been renamed to service.workflowsteps.datamodel and the default value has changed from MessageInfo to TransactionInfo.

7/ Instance Base Context and Swagger(OpenApi)

The Instance record created via the OAuth Settings portal menu requires an update.

The Base context of a p6 instance is now: http//hostname:8080/p6 (not http://hostname:8080/b2box/rest/resource)

Swagger support has been upgraded to V3 which is now a standard know as OpenAPI. The V1 Swagger user interface is no longer embedded in the product. The latest Swagger user interface can be downloaded from https://swagger.io/tools/swagger-ui/

The Swagger UI should use the endpoint: http://hostname:8080/p6/openapi

Note

At this stage, you can finally start your instance in version 6.0.0-alpha-5. Change the version in the Docker Compose file and start your instance.

DSL

fileFromUri DSL Variable Removed

Please use fileFromUrl from the uri DSL

DSL access

To access to the DSL you need to prefix them by p6., for example:

log.debug p6.counter.list()

Transaction DSL

This version makes TransactionInfo the default model for transactions rather than MessageInfo. Also, in order to ease P6 learning curve, the term ‘project’ was replaced with ‘save’, therefore the following functions were renamed:

  • putMI -> putTI
  • projectMI -> saveTI which validates against a TransactionInfo and not a MessageInfo
  • project -> save
  • p6projectAndRouteMI -> saveAndRouteTI which validates against a TransactionInfo and not a MessageInfo
  • p6projectAndRoute -> saveAndRoute
  • p6route -> route which is not to be confused with the old function route which used routing rules!

In P6 version 6.0.0 all support for the old routing rules was removed, hence the following functions disappeared:

  • projectAndRouteMI
  • projectAndRoute
  • route

DataModel

The autoloaded data model WFWorkItem has been renamed to WorkflowTask. Make sure to update your stored procedures, XSLT resources, scripts…

Integrations

Each Platform 6 instance should now be defined and configured as an Integration using the new Administration panel.

See: Adding Your Instance ‘Integration’

Routes

Web3j

The component is no longer called P6Web3j, because we now rely on the standard Web3j Camel component. Therefore, all routes with endpoint URI p6web3j://... should be renamed to web3j://....

The producer query parameter transactionIndex was renamed to indexto comply with the official Web3j Camel component.

Observable operations were renamed to align with Web3j version 4.3:

  • ETH_LOG_OBSERVABLE -> ETH_LOG_FLOWABLE
  • ETH_JSON_LOG_OBSERVABLE -> ETH_JSON_LOG_FLOWABLE
  • ETH_BLOCK_HASH_OBSERVABLE -> ETH_BLOCK_HASH_FLOWABLE
  • ETH_PENDING_TRANSACTION_HASH_OBSERVABLE -> ETH_PENDING_TRANSACTION_HASH_FLOWABLE
  • TRANSACTION_OBSERVABLE -> TRANSACTION_FLOWABLE
  • PENDING_TRANSACTION_OBSERVABLE -> PENDING_TRANSACTION_FLOWABLE
  • BLOCK_OBSERVABLE -> BLOCK_FLOWABLE
  • REPLAY_BLOCKS_OBSERVABLE -> REPLAY_PAST_BLOCKS_FLOWABLE
  • REPLAY_TRANSACTIONS_OBSERVABLE -> REPLAY_PAST_TRANSACTIONS_FLOWABLE
  • CATCH_UP_TO_LATEST_BLOCK_OBSERVABLE -> REPLAY_ALL_PAST_BLOCKS_FLOWABLE
  • CATCH_UP_TO_LATEST_TRANSACTION_OBSERVABLE -> REPLAY_ALL_PAST_TRANSACTIONS_FLOWABLE
  • CATCH_UP_TO_LATEST_AND_SUBSCRIBE_TO_NEW_BLOCKS_OBSERVABLE -> REPLAY_PAST_AND_FUTURE_BLOCKS_FLOWABLE
  • CATCH_UP_TO_LATEST_AND_SUBSCRIBE_TO_NEW_TRANSACTIONS_OBSERVABLE -> REPLAY_PAST_AND_FUTURE_TRANSACTIONS_FLOWABLE

Pipeline Variables

The following automatically filled pipeline variables were renamed: - platform6.request.cluster -> platform6.request.dataPartition - platform6.request.concept -> platform6.request.dataType

Routing Order Attribute Changes

Routing orders were designed to store calls to adapters and their parameters. This was later extended to also include service calls via the svc:\\ service URL syntax.

Since adapters have been completely removed in Platform 6, the Routing Order attributes: Adapter and Adapter Parameters have been re-purposed to Service and Service Parameters

Warning

The svc:\\ URL syntax is no longer supported.

The Service attribute must now contain the name of the target service. For example platform6.scripts (or scripts for short)

The Service Parameters attribute now contains a comma separated list of URL encoded name/value pairs

The change to format and use of attributes means that any routing orders created prior to this release will not be able to be reprocessed ( Unless they are first heavily edited first using the Routing Orders UI.)

Windows support

Installing Platform 6 natively on Windows is no longer supported. If you want to use Platform 6 on Windows it has to be done using Docker through the Platform 6 install scripts on Github.

Dependency Upgrades

  • Bouncy Castle from 1.59 to 1.61
  • Flying Saucer PDF from 9.0.7 to 9.1.18
  • Groovy from 2.5.6 to 2.5.7
  • Web3j from 3.5.0 to 4.3.1

log4j.properties

If you have made a copy of the log4j.properties file and stored it as an override in your instance p6core_data/conf folder you must update it.

The line:

log4j.appender.b2box = io.platform6.core.impl.logging.B2boxAppender

should be changed to

log4j.appender.platform6 = io.platform6.core.impl.logging.Platform6Appender

And the corresponding reference at the top of the file to appender b2box must be changed to platform6

log4j.rootCategory=ALL, platform6, rfile

Version 6.0.0-beta-1

ApplicationId vs. InstanceId

A terminology change during development of Platform 6 led to a confusing mix of terms where applicationId and instanceId where used interchangeably to refer the same thing!

We have now standardised on InstanceId

Warning

This effects a mandatory attribute in the application.conf of every instance

The application.conf attribute:

applicationid = myp6instance

must be changed to:

instance.id = myp6instance

before Platform 6 will start.

DSL

The package name containing the DSL has been renamed from com.amalto.b2box.groovy to io.platform6.core.dsl and the DSL are no longer accessible without using the prefix p6..

Annotations

The following annotations was renamed: - B2GroovyBinding -> P6GroovyBinding - B2ExecutionContext -> P6ExecutionContext - B2User -> P6User

and also moved from the package com.amalto.b2box.groovy.dsl.bindingto io.platform6.core.impl.application.scripting.annotations

Transaction DSL

The getSerialForm method has been removed

Table DSL

The following function was renamed: - upsertRecords -> upsert

Note: Existing scripts will be updated to use the new DSL function so no manual action is required.

Warning: If you are using P6Sync don’t forget to retrieve the online modification using the -pull option

Utils DSL

The following bindings has been moved (for both scripts and routes): - escapeXml -> p6.utils.escapeXml - unescapeXml -> p6.utils.unescapeXml - pause -> p6.utils.pause

Note: Existing scripts will be updated so no manual action is required.

Warning: If you are using P6Sync don’t forget to retrieve the online modification using the -pull option

Routes

The following bindings has been moved for route definition: - utils -> p6.utils - configuration -> p6.configuration - camel -> p6.camel - service -> p6.service - appconfig -> p6.appconfig

Note: Existing routes will be updated so no manual action is required.

Warning: If you are using P6Sync don’t forget to retrieve the online modification using the -pull option

Exception

The use of io.platform6.core.api.B2boxException has been replaced by io.platform6.common.util.P6Exception.

Note: Existing routes will be updated to use the new exception so no manual action is required.

Warning: If you are using P6Sync don’t forget to retrieve the online modification using the -pull option

Configuration files

The following file was renamed: - b2boxcontext.xml -> p6context.xml

The configuration entries starting with b2audit. have been renamed to audit.. And configuration entries starting with b2auth. have been renamed to p6auth..

Log appender renamed

io.platform6.core.impl.logging.B2boxAppender was renamed to io.platform6.core.impl.logging.Platform6Appender.

Services

Transactions

The endpoint [DELETE] /apis/v2/platform6.transactions/transaction/{view}/{ids:.*} has been removed. You should use the endpoint [DELETE] /apis/v2/platform6.transactions/transactions instead.

Endpoint permissions

The endpoints to deal with file submission have change and you need the following permissions:

Service createSession, addFile deleteFile, deleteSession listFiles
transactions [transactions=edit] [transactions=delete] [transactions=read]
bundledresources [bundledresources=edit] [bundledresources=delete] [bundledresources=read]
workflowsteps [workflowsteps=edit] [workflowsteps=delete] [workflowsteps=read]

Version 6.0.0-beta-2

Endpoint permissions

Endpoint permissions have been simplified and the ones that have been modified are listed bellow.

REST API Old permissions New permissions
Add an item SERVICE_ID=read && SERVICE_ID=edit SERVICE_ID=edit
Delete an item SERVICE_ID=read && SERVICE_ID=edit SERVICE_ID=read && SERVICE_ID=delete
Delete items SERVICE_ID=read && SERVICE_ID=edit SERVICE_ID=read && SERVICE_ID=delete
Export items SERVICE_ID=read && SERVICE_ID=edit SERVICE_ID=read

Applications

The following services will be automatically restarted following application installation and un-installation (assuming the application contains service items deployed against each service)

  • Workflow Steps
  • Routes
  • Transactions

No manual action is required, except cleaning up your install and uninstall scripts in your applications from calls that restart the above services.

Configuration

All the following configuration keys have been replaced by p6.realm:

  • groovy.b2.realm
  • default.p6.realm
  • b2box.b2.realm
  • service.workflow.b2.realm

The following configuration entries have been renamed:

  • counters.* -> p6.service.counters.*
  • service.workflow.* -> p6.service.workflowsteps.*
  • service.* -> p6.service.*
  • com.amalto.service.<name.serviceName>.limiter.* -> p6.service.<name>.limiter.*
  • io.platform6.core.impl.application.camel.p6log.LogConsumer -> p6.camel.p6log.LogConsumer
  • audit.* -> p6.service.auditing.*
  • store.* -> p6.store.*
  • swagger.ui -> p6.swagger.ui
  • io.platform6.core.impl.platform.jobcontrol.JobControlManagerImpl.* -> p6.job.*
  • job.control.execution.messages.length -> p6.job.control.execution.messages.length
  • disable.get.cache.headers -> p6.cache.annotation
  • cmb.service.runner.pool.size.<name> -> p6.service.<name>.runner.pool.size

DSL Changes

The method defaultTransactionManager in the ethereumrpc DSL was renamed to noOpTransactionManager.

The DSL p6.control was renamed to p6.script. The method moduleNames in the script DSL was renamed to list.

Note

The renaming is automatically handled in scripts, no manual action is required.

The method getCompiled in the resource DSL was removed.

Version 6.0.0

Before the migration

The default redirect URI for OAuth clients was changed from http://www.b2box.com to http://www.platform6.io. Therefore if your instance still uses the default b2box client to connect to P6 Auth, add the following line to application.conf:

p6auth.client.redirect.uri="http://www.b2box.com"

Variables

The following variables have been renamed in the application: - B2BOX_DATA -> P6_DATA - B2BOX_HOME -> P6_HOME

This affects: - Scripts (main script and resources) - Routes (template and script segments) - URL (any use of an URL starting with file:) - Bundled resources (target root) - Workflow steps (URL inside the definition of thr xml step)

Note

The renaming is automatically handled in the previous cases by SQL migration scripts

Use official Camel Web3j routes

This release is aligned with the official Camel Web3j module. Therefore many custom headers that were added in previous versions were removed. Here is the list of removed headers:

  • logAddress
  • logBlockHash
  • logBlockNumber
  • logData
  • logIndex
  • logTransactionHash
  • logTransactionIndex
  • logType
  • logTopics

These headers are the properties of a org.web3j.protocol.core.methods.response.Log that were extracted. To migrate your blockchain listener routes, you need to serialize every incoming Log from the blockchain to JSON, then parse the JSON string and read the property you want. A full example of such a route is provided in the Demo App.

Version 6.0.1

Before the migration

Make sure to change volume mapping in Docker to INSTANCE_DATA_PATH/p6core.data:/opt/p6core.data (rather than b2box5).

DSL Changes

After all enhancements made to the AppConfig DSL, the Ethereum DSL no longer relies on the Tables DSL to store credentials. Therefore, the following functions were removed:

  • String addNewCredentials(String tableName, Map keys, String fieldName, String password, boolean useFullScrypt)
  • Credentials getCredentials(String tableName, Map keys, String fieldName, String password)

Some Changes To Defaults

  • The default p6auth.client.id in reference.conf is no longer b2box. It is now p6core. You may need to change you current Client definition using OAuth setting or switch to the more current Integrations way of integrating p6core instances with P6 Auth.
  • The default log file name has changed in log4j.properties from b2box.log to p6core.log.
  • The default install location of the product within the docker container has changed from /opt/b2box to /opt/p6core.
  • The default TMP folder used within the docker container has changed from /tmp/b2box to /tmp/p6core.

New JVM

For licencing reasons and for future independence we have stopped using the Oracle distribution of Java. We now use the official OpenJDK provided Docker container image:

  • openjdk:8-jre-alpine

This is a significant change. Not only are we moving away from the Oracle distribution, but we are also moving from a JDK to a JRE. In combination with the Alpine linux distribution the overall size of the P6 Core container has been significantly reduced.

Note: We no longer extend the original b2base image that contained the S6 service manager. We no longer configure the postfix service or run P6 Core as a service.

Services

Bundled resources

Endpoints

The endpoint [DELETE] /apis/v2/platform6.bundledresources/bundledresource/{name}/{appKey: .*} has been removed. You should use the endpoint [POST] /apis/v2/platform6.bundledresources/bundledresources/delete instead.

The endpoint [PUT] /apis/v2/platform6.bundledresources/bundledresource has been removed. You should use the endpoint [PUT] /apis/v2/platform6.bundledresources/bundledresources/{sessionToken: .*} instead. The body of the request has to be the BundledResource item and the sessionToken can be empty.

The endpoint [POST] /apis/v2/platform6.bundledresources/bundledresource has been removed. You should use the endpoint [POST] /apis/v2/platform6.bundledresources/bundledresources/{sessionToken: .*} instead. The body of the request has to be the BundledResource item and the sessionToken can be empty.

Version 6.0.3

Before the migration

All class packages were renamed from com.amalto.b2box... to io.platform6.... Therefore make sure class names are correct in your properties files in the conf folder. For instance, com.amalto.b2box.core.store.impl.dialect.pgsqlxml.mappers.proc.ReportingDataSourceProcMapper was renamed to io.platform6.core.store.impl.dialect.pgsqlxml.mappers.proc.ReportingDataSourceProcMapper.

Pipeline Changes

The content type of variables injected in the pipeline changed as follows:

  • application/b2box.itempk -> application/p6core.itempk
  • application/b2box.dom -> application/p6core.dom
  • application/b2box.i18n -> application/p6core.i18n

Note

Renaming is automatically handled by SQL migration scripts, no manual action required.

Version 6.0.6

Services

Audit

The configuration keys moved from audit to p6.audit.

Only the following providers are now available:

  • LOG4J
  • ELASTICSEARCH

Example: 

p6.service.auditing.provider=ELASTICSEARCH

Version 6.1.0

DSL

ethereumrpc

The name of this DSL has been changed to web3ethereum.

Note

Renaming is automatically handled by SQL migration scripts, no manual action required.

Version 6.1.1

DSL

Log

The log binding inside the scripts is now deprecated and calls should be replaced by p6.log.

The p6.log binding is also available inside route definitions.

Note

The old binding is kept for backward compatibility but will be removed in a next release.

Service

Workflow step

The use of the node <AllowMessageEdit> was deprecated and has been removed. You must now use the node <AllowTransactionEdit> instead as describe in the workflow step configuration

Note

This is automatically handled by SQL migration scripts, no manual action required.

When invoking a workflow step, the header of the request named platform6.request.dataPartition is now deprecated and can be removed. If set, the value will be ignored.

Version 6.2.1

Database

The table item has been removed. Data are now split in dedicated tables: - log - table_data - transaction

On each table the column datapartition has also been removed.

The table_data table has new iid columns: - iid5 - iid6 - iid7 - iid8

Version 6.3.3

Monitoring

The following monitoring endpoints have been removed: - http://:8080/p6/vmstats - http://:8080/p6/log4jstats

Sensu alerts need to be migrated to target the “/p6” endpoint which returns 200 OK to signal that the instance is up and running.

Views

The <Searchable> type values have been renamed: - IStringEquals -> CaseInsensitiveString - StringEquals -> String - StringContains -> Words

Note

This is automatically handled by SQL migration scripts, no manual action required.

Dependencies

Saxon-b

The saxon-b dependency has been removed from Platform 6’s code base.

If you still need Saxon-b either add it using the Bundled Resources service or add the JAR to the classpath manually.

Warning

Due to a security issue it is not recommended to use that dependency. You should migrate to Saxon-HE that is already shipped inside Platform 6.

XML/XSL security

Explanation

Protections have been added to prevent XXE attacks in XML and XSL files.

The following parameters have been disabled:

  • external general entities
  • external parameter entities
  • external dtd
  • XInclude markup

If an XSL file uses a reference to a file or a URL 

<!DOCTYPE transform [<!ENTITY xxe SYSTEM "file:///etc/passwd" >]>
<xsl:transform xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0">
<xsl:template match="/">
...

a P6Exception will be thrown Error reported by XML parser processing null: URI scheme 'file' has been disallowed

If an XML file uses a reference to a file or a URL, the content of the variable will be an empty string.

Impact

Workflow step WorkItem node

The node <WorkItem> (not the one under <ViewNames>) has been renamed to <WorkflowTaskEnhancer>

Note

This is automatically handled by SQL migration scripts, no manual action required.

Workflow step Parameters node

Schema validation has been added to the Workflow Step items. Exceptions will be thrown while trying to save invalid XML definitions.

The nodes <Parameter> have to be wrapped inside a new <Parameters> node.

Before 

<WorkflowStep>
    <Actions>
        <Parameter>...</Parameter>
        <Parameter>...</Parameter>
    </Actions>
</WorkflowStep>

After 

<WorkflowStep>
    <Actions>
        <Parameters>
            <Parameter>...</Parameter>
            <Parameter>...</Parameter>
        </Parameters>
    </Actions>
</WorkflowStep>

Warning

This change has to be done manually because it cannot be automated!

Version 6.4.0

Database

Since the version 12, PostgreSQL integrates a new progress view for index creation.

Note

Although not mandatory, it’s highly recommended migrating to the latest version of PostgreSQL

A new index management has been implemented for Views and TableData. Now, when editing one of those items you can generate all the indexes needed with one click. However, it will not deal with existing indexes because the process has its own naming convention.

Example (for table data):

  • before: xraxsszuvdomvnag6s
  • after: idx_p6_demo_items_8xknhcfpft7lvxbt

Example (for transactions):

  • before: item_ti_fts_idx01
  • after: idx_p6_demo_transactions_c6gnmiidbbam2fij

To avoid duplicate indexes and unused indexes, please remove all previous indexes that are not required anymore after running the index creation for the first time.

Note

For items bundled with an application, indexes are created automatically at application installation. Indexes are also dropped when uninstalling the application.

Configuration

The configuration key p6.realm has been replaced by p6auth.realm.default provided by P6 Auth.

No need to add it in application.conf unless you want to override the value of P6 Auth.