Ensure to perform complex column updates only when supported

[sgiehl]

Description:

Some database engines, like TiDb, are performing sanity checks for table updates. Those might include checking if all columns used already exist. In such a case a query that adds a new column and e.g. directly uses it for an index will fail.

This PR changes how dimension updates are performed, so queries are executed separately instead of combined, if the database doesn't support it.

The same is applied for the AddColumns database migration class. It allows adding multiple columns, while every column will be placed after the one added before. This might also fail, so each column needs to be added separately

Review

• Functional review done
• Potential edge cases thought about (behavior of the code with strange input, with strange internal state or possible interactions with other Matomo subsystems)
• Usability review done (is anything maybe unclear or think about anything that would cause people to reach out to support)
• Security review done
• Wording review done
• Code review done
• Tests were added if useful/possible
• Reviewed for breaking changes
• Developer changelog updated if needed
• Documentation added if needed
• Existing documentation updated if needed

[michalkleiner]

The addition on its own makes sense, but do we need two ways of writing such update queries? Can we live with the simple approach only where each column, each index needs to be added separately? Would it be such a huge performance penalty for database engines that do support the complex queries?

[sgiehl]

@michalkleiner We can't ensure that it won't cause performance problems when executing the queries one by one for all database engines. Adding or changing columns and indices, can always cause a full index rebuilt. When executed as one query, the database might only rebuilt the index once, as it performs all other changes before. In separate queries it might cause an index to be rebuilt multiple times, which can take ages depending on the table and data included.

[michalkleiner]

Good point, the index rebuild didn't cross my mind.

[michalkleiner]

Do we have some tests for the Updater to test the mechanism?

[sgiehl]

The are some tests covering that partially. The Installation only sets up the base schema and afterwards all dimension changes / updates will be performed. That should be covered by the installation UI tests.
In addition there is a UI test, that performs an update from an old Matomo 1.6 database. That should cover the change for the migration.
But both for sure currently only cover the case when complex updates are supported (here). The other case will be covered at some point by the TiDb PR.