Migrate your App data with Jira Cloud Migration Assistant

Hello!

This article will show you how you can migrate your data with Jira Cloud Migration Assistant (JCMA).

JCMA helps you migrate Jira Software and Jira Core data from Server or Data Center to Cloud.

When you develop your app for Jira Server, you can store your data in multiple places: issue properties, project properties, Jira database using Active Objects, file system. JCMA will migrate data from issue properties, but it will not be able to migrate data from AO tables or file systems. In this case, your app will not be able to function correctly in Cloud after migration.

To migrate all the data for your Jira Server app to Cloud correctly, you can extend JCMA to migrate your data. You can find more information about it here, and you can find example source code for extensions here.

This article will explain how it all works in detail with a ready for use code. You can see this code here.

You can also watch a video over here.

Theory

JCMA extension app consists of two apps: one for Jira Server/Data Center and one for Jira Cloud.

Migration starts in Jira Server/Data Center. After JCMA has finished with its data migration, JCMA executes all classes which implement the CloudConnectionListener interface. And that is what we have to do, and we need to create a class that implements this interface in our Jira Server app.

Then JCMA sends events to migration webhooks registered for the Jira Cloud instance where we migrate data. In our Jira Cloud app, we need to register our webhooks which the events will trigger.

Ok. That is all for theory. Let’s have a look at the code.

Jira Server/Data Center app

We have two classes.

src/main/java/ru/matveev/alexey/atlassian/server/accessor/LocalCloudMigrationAccessor.java

@Named("cloudMigrationAccessor")
public class LocalCloudMigrationAccessor implements ApplicationContextAware {

    private CloudMigrationAccessor registrar;

    @Override
    public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
        try {
            this.registrar = (CloudMigrationAccessor) applicationContext.getAutowireCapableBeanFactory().
                    createBean(getClass().getClassLoader().loadClass("com.atlassian.migration.app.tracker.CloudMigrationAccessor"), AutowireCapableBeanFactory.AUTOWIRE_CONSTRUCTOR, false);
        } catch (Exception e) {
            throw new RuntimeException("Failed to initialise CloudMigrationAccessor", e);
        }
    }

    public CloudMigrationAccessor getCloudMigrationAccessor() {
        return registrar;
    }
}

This class gets a reference to the CloudMigrationAccessor service. We need this service to execute our migration.

src/main/java/ru/matveev/alexey/atlassian/server/impl/MyPluginComponentImpl.java

@Slf4j
@Named ("myPluginComponent")
public class MyPluginComponentImpl implements CloudMigrationListener, InitializingBean, DisposableBean {

    private final CloudMigrationAccessor accessor;

    @Inject
    public MyPluginComponentImpl(LocalCloudMigrationAccessor accessor) {
        // It is not safe to save a direct reference to the gateway as that can change over time
        this.accessor = accessor.getCloudMigrationAccessor();
    }

    @Override
    public void onRegistrationAccepted() {
        log.info("Nice! The migration listener is ready to take migrations events");
    }

    /**
     * Just a collection of example operations that you can run as part of a migration. None of them are actually required
     */
    @Override
    public void onStartAppMigration(String transferId, MigrationDetails migrationDetails) {
        ObjectMapper objectMapper = new ObjectMapper();
        try {
            log.info("Migration context summary: " + objectMapper.writeValueAsString(migrationDetails));
            PaginatedMapping paginatedMapping = accessor.getCloudMigrationGateway().getPaginatedMapping(transferId, "identity:user", 5);
            while (paginatedMapping.next()) {

                Map<String, String> mappings = paginatedMapping.getMapping();
                log.info("mappings = {}", objectMapper.writeValueAsString(mappings));
            }
        } catch (IOException e) {
            log.error("Error while running app migration", e);
        }

        // You can also upload one or more files to the cloud. You'll be able to retrieve them through Atlassian Connect
        try {
            OutputStream firstDataStream = accessor.getCloudMigrationGateway().createAppData(transferId);
            // You can even upload big files in here
            firstDataStream.write("Your binary data goes here".getBytes());
            firstDataStream.close();

            // You can also apply labels to distinguish files or to add meta data to support your import process
            OutputStream secondDataStream = accessor.getCloudMigrationGateway().createAppData(transferId, "some-optional-label");
            secondDataStream.write("more bytes".getBytes());
            secondDataStream.close();
        } catch (IOException e) {
            log.error("Error uploading files to the cloud", e);
        }
    }

    @Override
    public void onRegistrarRemoved() {
        log.info("The listener is no longer active");
    }

    @Override
    public String getCloudAppKey() {
        return "ru.matveev.alexey.atlassian.server.jcma-cloud";
    }

    @Override
    public String getServerAppKey() {
        return "ru.matveev.alexey.atlassian.server.jcma-server";
    }

    @Override
    public Set getDataAccessScopes() {
        return Stream.of(
                AccessScope.APP_DATA_OTHER,
                AccessScope.PRODUCT_DATA_OTHER,
                AccessScope.MIGRATION_TRACING_IDENTITY)
                .collect(Collectors.toCollection(HashSet::new));
    }

    @Override
    public void afterPropertiesSet() {
        this.accessor.registerListener(this);
    }

    @Override
    public void destroy() {
        this.accessor.deregisterListener(this);
    }
}

Here we implement our class from CloudMigrationListener, register and unregister our listener in the afterPropertiesSet and destroy methods.

The onRegistrationAccepted method is called by JCMA after our listener was successfully registered by JCMA.

The getCloudAppKey and getServerAppKey methods return the keys of our apps in Server and Cloud.

And the onStartAppMigration method is the method called when JCMA wants your app to migrate data.

 @Override
    public void onStartAppMigration(String transferId, MigrationDetails migrationDetails) {
        ObjectMapper objectMapper = new ObjectMapper();
        try {
            log.info("Migration context summary: " + objectMapper.writeValueAsString(migrationDetails));
            PaginatedMapping paginatedMapping = accessor.getCloudMigrationGateway().getPaginatedMapping(transferId, "identity:user", 5);
            while (paginatedMapping.next()) {

                Map<String, String> mappings = paginatedMapping.getMapping();
                log.info("mappings = {}", objectMapper.writeValueAsString(mappings));
            }
        } catch (IOException e) {
            log.error("Error while running app migration", e);
        }

        // You can also upload one or more files to the cloud. You'll be able to retrieve them through Atlassian Connect
        try {
            OutputStream firstDataStream = accessor.getCloudMigrationGateway().createAppData(transferId);
            // You can even upload big files in here
            firstDataStream.write("Your binary data goes here".getBytes());
            firstDataStream.close();

            // You can also apply labels to distinguish files or to add meta data to support your import process
            OutputStream secondDataStream = accessor.getCloudMigrationGateway().createAppData(transferId, "some-optional-label");
            secondDataStream.write("more bytes".getBytes());
            secondDataStream.close();
        } catch (IOException e) {
            log.error("Error uploading files to the cloud", e);
        }
    }

As you can see, there are two parameters passed to the function: transferId and migrationDetails. The transferId parameter helps your Server and Cloud app work with the same migrated data. You can run multiple migrations by JCMA, and transferId is the id for those migrations. The migrationDetails contain migration context. I want to pay attention to the fact that you can get mappings in this method. Mappings provide mapping for the server object id with the cloud object id.

For example, let’s say you migrated users. In Jira Server, the user is called admin, but in Cloud, it will have a different id. And by mappings, you can retrieve this new id. Also, you can retrieve ids for such objects as issues, workflows, issue types, etc.

So in our implementation of the onStartAppMigration method, we create two files, which will be put into Cloud storage by JCMA for us and later read in our Cloud app. Those files should contain data that should be migrated to Cloud in actual applications.

Of course, we could start and finish migration in the onStartAppMigration method itself. For example, let’s say we have REST API in our Cloud app, which will accept our data and put everything as it should be in Cloud. We could call this REST API from this method and pass all data we are migrating. But it is not always the case; I made this example more complicated and created a Jira Cloud app that will be triggered upon migration.

Jira Cloud app

First, we need to tell Atlassian Migration API which of our endpoints to call upon migration. In other words – to register our webhooks. I will register our webhooks in the installed lifecycle event, and I will unregister those webhooks in the uninstalled lifecycle event.

I use atlassian-connect-spring-boot. That is why I need to implement two listeners for those lifecycles events.

But to register my webhooks, I need to know the base URL of my app, and I will get it from the application.yml file with src/main/java/ru/matveev/alexey/atlassian/cloud/util/YAMLConfig.java:

@Configuration
@EnableConfigurationProperties
@ConfigurationProperties(prefix="addon")
@Data
public class YAMLConfig {

    private String key;
    private String baseUrl;


}

The code is simple.

Then I can create my listeners.

src/main/java/ru/matveev/alexey/atlassian/cloud/listener/AddonInstalledEventListener.java

@Component
@Slf4j
public class AddonInstalledEventListener implements ApplicationListener {
    @Autowired
    private YAMLConfig yamlConfig;
    @Autowired
    private AtlassianHostRestClients atlassianHostRestClients;

    @Override
    public void onApplicationEvent(AddonInstalledEvent addonInstalledEvent) {
        log.info("install event");
        JSONArray endpoints = new JSONArray();
        endpoints.put(String.format("%s/migration-started" , yamlConfig.getBaseUrl()));
        JSONObject webhook = new JSONObject();
        webhook.put("endpoints", endpoints);
        HttpHeaders headers = new HttpHeaders();
        headers.setContentType(MediaType.APPLICATION_JSON);
        HttpEntity request = new HttpEntity<>(webhook.toString(), headers);
        atlassianHostRestClients
                .authenticatedAsAddon()
                .put(String.format("%s/rest/atlassian-connect/1/migration/webhook", addonInstalledEvent.getHost().getBaseUrl()), request);

    }
}

As you can see, I inject YamlConfig and AtlassianHostRestClients. YamlConfig will give me the base URL of my application, and AtlassianHostRestClient will let me call Jira Cloud Rest Api with proper authentication.

The uninstalled event has the same code, and I will not paste it here.

So we registered our webhook. Now let’s write the endpoint for our webhook src/main/java/ru/matveev/alexey/atlassian/cloud/listener/MigrationStartedListener.java:

@Controller
@Slf4j
public class MigrationStartedListener {
    @Autowired
    private AtlassianHostRestClients atlassianHostRestClients;

    @PostMapping("/migration-started")
    @ResponseBody
    public String migrationStarted(@RequestBody JiraMigration body) {
        log.info("migration started");
        if ( "APP_DATA_UPLOADED".equals(body.getWebhookEventType())) {
            ResponseEntity response = atlassianHostRestClients.authenticatedAsAddon()
                    .getForEntity(String.format("%s/rest/atlassian-connect/1/migration/data/%s/all", body.getMigrationDetails().getCloudUrl(), body.getTransferId()), String.class);
            log.info(response.getBody());
        }
        return "ok";
    }
}

We defined the migration-started endpoint, and JCMA will call this endpoint. We accept the JiraMigration class as the body for the request, and this class contains valuable methods which will let us perform our migration. In my case, I get the tranferId, and after it, I get the files that I created in my Jira Server app.

That is a pretty simple example. But you can figure out how JCMA extensions work.