Export / Import WordPress site to / from a .zip file

[adamziel]

Blueprints library needs to support full WordPress import to support a) the importSite step, and b) arbitrary WordPress zips as the "base image". These explorations will inform the development of the Site Transfer Protocol.

Some related work was already dome in the WordPress Playground plugin (release link),

Discussion

• Importing does not attempt to merge the incoming site into the existing one, but completely replaces it.
• Exporting = Stream-compressing site files and the database data. Challenge: How to handle PHP timeouts? Idea: Don't handle them in v1. An export state table combined with exporting multiple zip files could resolve this problem later on.
  • Ensure binary data does not mess up the export file. Should it be base64-encoded?
• Importing = Stream-decompressing the exported zip and importing the database.
• All site URLs in the database, wp-config.php, wp-content, sunrise.php, mu-plugins are replaced with https://playground.internal when streaming the export, and replaced again with the target site domain when streaming the import. There's no database post processing.
• If the database is MySQL, it is serialized into an SQL file as seen in the WordPress Playground plugin.
• If the database is SQLite, should it be just transmitted as a binary blob?
  • The target WordPress environment might not have the SQLite extension installed. Perhaps both an .sql and an .sqlite file could be exported, and the target would choose which one to use?
  • There are some project ideas to store uploads or even all WordPress files in SQLite, so serializing that to SQL might make very little sense.
  • How would stream-replacing the URLs in a binary .sqlite file work? Is there any way of single-pass stream-processing it, or would it have to be post-processed after all? Could it be stream-processed by creating a new SQLite database and running SELECT * FROM and then INSERT INTO on each table and row? How would that be different from just exporting an SQL file?
• This repository provides stream handlers for ZIP files and HTTP requests they should be useful in this project.

Open questions

• Should env variables be exported?
• How should the database credentials be set? When importing into Playground, that isn't an issue. What should happen when importing into hosted WordPress?
• How to export a SQLite-based snapshot from a Playground and import it into a MySQL-based hosted WordPress? Or SQLite-based hosted WordPress? And vice versa?

Related tasks

• Bring this export implementation and this import implementation over from the WordPress Playground plugin as a starting point.
• Implement WordPress\Zip\Encoder based on this TypeScript zip encoder. Put it in WordPress\Zip. Replace ZipStream in the above with the new encoder.
• Export: Stream-replace site URLs in the WordPress files and SQL files as they are zipped.
• Import: Stream-replace site URLs in the WordPress files and SQL files as they are unzipped.
• If the database will be transmitted as an .sql file, the sqlite-database-integration will need to support SHOW CREATE TABLE query. Otherwise the WordPress Playground plugin will, ironically, not work in Playground.
• Replace the custom TypeScript import/export implementation built for https://playground.wordpress.net/ with the one shipped in the Blueprints library.
• Update the WordPress Playground plugin to use this library instead of shipping its own import/export implementation.
• Playground buffers the response data. Will streaming be required for this project?

Related resources

• https://core.trac.wordpress.org/ticket/60375
• Sync Protocol wordpress-playground#797
• Snapshots Support #9

cc @dmsnell